Python: how to use each line at a time from array in requests

2022-05-04 10:47 Johnny-b-13 imported from Stackoverflow

How to use each line at a time from an array further in requests:

The code:

x = ['param1', 'param2']
r = requests.get('http://somesite.com/?p='.format(x))

"https://www.w3schools.com/python/python_for_loops.asp" - Not working - only takes the last word from array

It works only for PRINT

2 answers

answered 2022-05-04 10:55 mteam88

Use a python for loop like this:

inputs = ["p1", "p2"]
result = []
for elem in inputs:
  result.append(requests.get(f'http://somesite.com/?p={elem}'))
print(result)

This should do what you want.

answered 2022-05-04 10:55 mak47
```
x = ['param1', 'param2']

for z in x
    r = requests.get('http://somesite.com/?p='.format(x))
    print(r)
```
for z in x is a for loop.

z could be anything, it's a placeholder for "one object in my array".

Your array has two items param1 and param2 so this will do whatever is inside that loop, in this case print them both once.

See also questions close to this topic

Python File Tagging System does not retrieve nested dictionaries in dictionary

I am building a file tagging system using Python. The idea is simple. Given a directory of files (and files within subdirectories), I want to filter them out using a filter input and tag those files with a word or a phrase.

If I got the following contents in my current directory:

data/
   budget.xls
   world_building_budget.txt
a.txt
b.exe
hello_world.dat
world_builder.spec

and I execute the following command in the shell:

py -3 tag_tool.py -filter=world -tag="World-Building Tool"

My output will be:

These files were tagged with "World-Building Tool":
data/ world_building_budget.txt
hello_world.dat
world_builder.spec

My current output isn't exactly like this but basically, I am converting all files and files within subdirectories into a single dictionary like this:

def fs_tree_to_dict(path_):
    file_token = ''
    for root, dirs, files in os.walk(path_):
        tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs}
        tree.update({f: file_token for f in files})
        return tree

Right now, my dictionary looks like this: key:''.

In the following function, I am turning the empty values '' into empty lists (to hold my tags):

def empty_str_to_list(d):
    for k,v in d.items():
        if v == '':
            d[k] = []
        elif isinstance(v, dict):
            empty_str_to_list(v)

When I run my entire code, this is my output:

hello_world.dat ['World-Building Tool']
world_builder.spec ['World-Building Tool']

But it does not see data/world_building_budget.txt. This is the full dictionary:

{'data': {'world_building_budget.txt': []},  
'a.txt': [],
'hello_world.dat': [], 
'b.exe': [], 
'world_builder.spec': []}

This is my full code:

import os, argparse


def fs_tree_to_dict(path_):
    file_token = ''
    for root, dirs, files in os.walk(path_):
        tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs}
        tree.update({f: file_token for f in files})
        return tree


def empty_str_to_list(d):
    for k, v in d.items():
        if v == '':
            d[k] = []
        elif isinstance(v, dict):
            empty_str_to_list(v)


parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter)

parser.add_argument("--filter", action="store", help="keyword to filter files")
parser.add_argument("--tag", action="store", help="a tag phrase to attach to a file")
parser.add_argument("--get_tagged", action="store", help="retrieve files matching an existing tag")

args = parser.parse_args()

filter = args.filter
tag = args.tag
get_tagged = args.get_tagged
current_dir = os.getcwd()

files_dict = fs_tree_to_dict(current_dir)
empty_str_to_list(files_dict)

for k, v in files_dict.items():
    if filter in k:
        if v == []:
            v.append(tag)
            print(k, v)

        elif isinstance(v, dict):
            empty_str_to_list(v)

    if get_tagged in v:
        print(k, v)

Actaully i am working on a project and in it, it is showing no module name pip_internal plz help me for the same. I am using pycharm(conda interpreter

 File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\Scripts\pip.exe\__main__.py", line 4, in <module>
  File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_internal\__init__.py", line 4, in <module>
    from pip_internal.utils import _log

I am using pycharm with conda interpreter.

Looping the function if the input is not string

I'm new to python (first of all) I have a homework to do a function about checking if an item exists in a dictionary or not.

inventory = {"apple" : 50, "orange" : 50, "pineapple" : 70, "strawberry" : 30}

def check_item():
    x = input("Enter the fruit's name: ")
    if not x.isalpha():
        print("Error! You need to type the name of the fruit")
    elif x in inventory:
        print("Fruit found:", x)
        print("Inventory available:", inventory[x],"KG")
    else:
        print("Fruit not found")
check_item()

I want the function to loop again only if the input written is not string. I've tried to type return Under print("Error! You need to type the name of the fruit") but didn't work. Help

How to get pass an array through a function with a user input?

I want to ask the user for the size of a 2D array arr[][], but also pass it through the function initializeArray. However, if I pass it through the function, I would have to have a size declarator for col, which doesn't allow the user to enter their own value for the size

  #include<iostream>
  using namespace std;

  void initializeArray(arr[][10], int N);

  int main() {
      int N;
      cout << "enter an array size: ";
      cin >> N;
      int arr[N][N];

      initializeArray(arr, N); // I get an error here
      for(int i = 0; i < N; i++) {
          for(int j = 0; j < N; j++)
              cout << arr[i][j] << " ";
          cout << endl;
      }
 }

  void initializeArray(int arr[][10], int N) {
      for(int i = 0; i < N; i++)
          for(int j = 0; j < N; j++)
              arr[i][j] = 0;
  }

The only solution I found was the make arr[][] a global array, but in that case, I would have to still declare the size parameters, and I want the user to enter whatever they want. Is there another way to fix this?

Two-dimensional array C++

Problem: Write a program of a two dimensional integer array of exactly four rows and four columns, and find the sum of the integers each row of the array as well as determine the smallest inputted element/data in the array. The user will input the elements. Display the elements in tabular form. Make the program user friendly. Use a Class and a member function for the process involve.

Pass Associative Array from Symfony Controller to Twig and JavaScript

So, I have 2 arrays that I've combined into one array after making sure they're the same length as such.

//Fetching Ratings for All Coaches & Making a Key, Value Associative Array with Coach id as Key and Rating as Value
        $coaches = $coachRepository->findAll();
        $rating = $coachRepository->findRatingByCoach();
        //Array of coach IDs
        $id_array = array();
        foreach ($coaches as $c){
            $id_array[] = $c->getId();
        }
        $combined = array_combine($id_array, $rating);

Now I want to pass this key, value array to Twig template and JavaScript. I'm currently doing this

    return $this->render('...', [
        ..., 'combined'=>$combined
    ]);

And I'm accessing the array in Twig as such

            {% for key, value in combined %}
                {{ key }} - {{ value }}
            {% endfor %}

And I'm getting this error.

Object of class App\Entity\Coach could not be converted to string

Looking at the Stack Trace I can see that the array combined is being passed in this form.

'combined' => array(object(Coach))

After a bit of debugging it turns out that I thought the issue was with the $id_array when in fact it's the $rating variable that's presenting an issue. I'm not sure why but I'm getting a single float value. I fixed the issue by changing how I'm fetching the $rating, now instead of using a QueryBuilder to get it from the DB, I'm getting it from list of Coaches as such

 //Array of coach IDs and array of Ratings
        $id_array = array();
        $rating_array = array();
        foreach ($coaches as $c){
            $id_array[] = $c->getId();
            $rating_array [] = $c->getRating();
        }

Even if I set the array

Scraping .aspx page with Python yields 404

I'm a web-scraping beginner and am trying to scrape this webpage: https://profiles.doe.mass.edu/statereport/ap.aspx

I'd like to be able to put in some settings at the top (like District, 2020-2021, Computer Science A, Female) and then download the resulting data for those settings.

Here's the code I'm currently using:

import requests
from bs4 import BeautifulSoup

url = 'https://profiles.doe.mass.edu/statereport/ap.aspx'
with requests.Session() as s:
    s.headers['User-Agent'] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:100.0) Gecko/20100101 Firefox/100.0"
    r = s.get('https://profiles.doe.mass.edu/statereport/ap.aspx')
    soup = BeautifulSoup(r.text,"lxml")
    data = {i['name']:i.get('value','') for i in soup.select('input[name]')}
    
    
    data["ctl00$ContentPlaceHolder1$ddReportType"]="DISTRICT",
    data["ctl00$ContentPlaceHolder1$ddYear"]="2021",
    data["ctl00$ContentPlaceHolder1$ddSubject"]="COMSCA",
    data["ctl00$ContentPlaceHolder1$ddStudentGroup"]="F",
    
    p = s.post(url,data=data)

When I print out p.text, I get a page with title '\t404 - Page Not Found\r\n' and message

<h2>We are unable to locate information at: <br /><br '
 '/>http://profiles.doe.mass.edu:80/statereport/ap.aspxp?ASP.NET_SessionId=bxfgao54wru50zl5tkmfml00</h2>\r\n'

Here's what data looks like before I modify it:

{'__EVENTVALIDATION': '/wEdAFXz4796FFICjJ1Xc5ZOd9SwSHUlrrW+2y3gXxnnQf/b23Vhtt4oQyaVxTPpLLu5SKjKYgCipfSrKpW6jkHllWSEpW6/zTHqyc3IGH3Y0p/oA6xdsl0Dt4O8D2I0RxEvXEWFWVOnvCipZArmSoAj/6Nog6zUh+Jhjqd1LNep6GtJczTu236xw2xaJFSzyG+xo1ygDunu7BCYVmh+LuKcW56TG5L0jGOqySgRaEMolHMgR0Wo68k/uWImXPWE+YrUtgDXkgqzsktuw0QHVZv7mSDJ31NaBb64Fs9ARJ5Argo+FxJW/LIaGGeAYoDphL88oao07IP77wrmH6t1R4d88C8ImDHG9DY3sCDemvzhV+wJcnU4a5qVvRziPyzqDWnj3tqRclGoSw0VvVK9w+C3/577Gx5gqF21UsZuYzfP4emcqvJ7ckTiBk7CpZkjUjM6Z9XchlxNjWi1LkzyZ8QMP0MaNCP4CVYJfndopwFzJC7kI3W106YIA/xglzXrSdmq6/MDUCczeqIsmRQGyTOkQFH724RllsbZyHoPHYvoSAJilrMQf6BUERVN4ojysx3fz5qZhZE7DWaJAC882mXz4mEtcevFrLwuVPD7iB2v2mlWoK0S5Chw4WavlmHC+9BRhT36jtBzSPRROlXuc6P9YehFJOmpQXqlVil7C9OylT4Kz5tYzrX9JVWEpeWULgo9Evm+ipJZOKY2YnC41xTK/MbZFxsIxqwHA3IuS10Q5laFojoB+e+FDCqazV9MvcHllsPv2TK3N1oNHA8ODKnEABoLdRgumrTLDF8Lh+k+Y4EROoHhBaO3aMppAI52v3ajRcCFET22jbEm/5+P2TG2dhPhYgtZ8M/e/AoXht29ixVQ1ReO/6bhLIM+i48RTmcl76n1mNjfimB8r3irXQGYIEqCkXlUHZ/SNlRYyx3obJ6E/eljlPveWNidFHOaj+FznOh264qDkMm7fF78WBO2v0x+or1WGijWDdQtRy9WRKXchYxUchmBlYm15YbBfMrIB7+77NJV+M6uIVVnCyiDRGj+oPXcTYxqSUCLrOMQyzYKJeu8/hWD0gOdKeoYUdUUJq4idIk+bLYy76sI/N2aK+aXZo/JPQ+23gTHzIlyi4Io7O6kXaULPs8rfo8hpkH1qXyKb/rP2VJBNWgyp8jOMx9px+m4/e2Iecd86E4eN4Rk6OIiwqGp+dMdgntXu5ruRHb1awPlVmDw92dL1P0b0XxJW7EGfMzyssMDhs1VT6K6iMUTHbuXkNGaEG1dP1h4ktnCwGqDLVutU6UuzT6i4nfqnvFjGK9+7Ze8qWIl8SYyhmvzmgpLjdMuF9CYMQ2Aa79HXLKFACsSSm0dyiU1/ZGyII2Fvga9o+nVV1jZam3LkcAPaXEKwEyJXfN/DA7P4nFAaQ+QP+2bSgrcw+/dw+86OhPyG88qyJwqZODEXE1WB5zSOUywGb1/Xed7wq9WoRs6v8rAK5c/2iH7YLiJ4mUVDo+7WCKrzO5+Hsyah3frMKbheY1acRmSVUzRgCnTx7jvcLGR9Jbt6TredqZaWZBrDFcntdg7EHd7imK5PqjUld3iCVjdyO+yLKUkMKiFD85G3vEferg/Q/TtfVBqeTU0ohP9d+CsKOmV/dxVYWEtBcfa9KiN6j4N8pP7+3iUOhajojZ8jV98kxT0zPZlzkpqI4SwR6Ys8d2RjIi5K+oQul4pL5u+zZvX0lsLP9Jl7FeVTfBvST67T6ohz8dl9gBfmmbwnT23SyuFSUGd6ZGaKE+9kKYmuImW7w3ePs7C70yDWHpIpxP/IJ4GHb36LWto2g3Ld3goCQ4fXPu7C4iTiN6b5WUSlJJsWGF4eQkJue8=',
 '__VIEWSTATE': '/wEPDwUKLTM0NzY4OTQ4NmRkDwwPzTpuna+yxVhQxpRF4n2+zYKQtotwRPqzuCkRvyU=',
 '__VIEWSTATEGENERATOR': '2B6F8D71',
 'ctl00$ContentPlaceHolder1$btnViewReport': 'View Report',
 'ctl00$ContentPlaceHolder1$hfExport': 'ViewReport',
 'leftNavId': '11241',
 'quickSearchValue': '',
 'runQuickSearch': 'Y',
 'searchType': 'QUICK',
 'searchtext': ''}

Following suggestions from similar questions, I've tried playing around with the parameters, editing data in various ways (to emulate the POST request that I see in my browser when I navigate the site myself), and specifying an ASP.NET_SessionId, but to no avail.

How can I access the information from this website?

scraping NYT mini crossword stats gives 403 forbidden URL error

I'm trying to scrape my NYT mini crossword stats to then update a google sheet. But I'm having trouble with the login portion of the code. Here's my code so far:

from http import cookies
import pandas as pd
import requests
from bs4 import BeautifulSoup
import re

s = requests.session()

nyt_login_url = 'https://myaccount.nytimes.com/'

login_route = 'svc/lire_ui/login'

auth_token = 'my auth token from requests'

HEADERS = {
   'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.109 Safari/537.36',
   'origin': nyt_login_url,
   'referer': nyt_login_url + login_route
}

login_payload = {
    'login': 'myusername',
    'password': 'mypassword',
    'authtoken': auth_token
}

login_req = s.post(nyt_login_url + login_route, headers = HEADERS, data = login_payload)

print(login_req.raise_for_status())

When I run this I receive the following error:

requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://myaccount.nytimes.com/svc/lire_ui/login

If I successfully log in, the status code should be 200, but not sure how to get there. Instead, it's 403. Any help?

How to convert Curl SSL and auth request to Python code

I would like to convert my request written in Curl to Python. I'm ignoring ssl becouse this certificate is signed by me. I am not sure if I have correctly replaced the value of "description" from null to None. How to improve my query written in Python?

Curl

curl -k -i -u 'admin:admin' --header 'Content-Type: application/json' -d '{"name": "Testowa Rola 221", "description": null, "privilegeIds": [1, 2, 3], "userIds": []}' -X POST https://localhost:44411/api/Role

Python

import requests


url = 'https://localhost:44411/api/Role'
data = {
        "name": "Test name 12",
        "description": None,
        "privilegeIds": [1, 2, 3],
        "userIds": []
}
headers = {
    'Content-Type': 'application/json'
}
auth = (
    'admin',
    'admin'
)
verify = '/home/dominik/jpk/xefix/ssl/https.crt'

create_administrator = requests.post(url=url, verify=verify, headers=headers, data=data, auth=False)
print(create_administrator.text)

Related questions

Python: how to use each line at a time from array in requests

2 answers

See also questions close to this topic