Python: how to use each line at a time from array in requests
How to use each line at a time from an array further in requests:
The code:
x = ['param1', 'param2']
r = requests.get('http://somesite.com/?p='.format(x))
"https://www.w3schools.com/python/python_for_loops.asp" - Not working - only takes the last word from array
It works only for PRINT
2 answers
-
answered 2022-05-04 10:55
mteam88
Use a python
for
loop like this:inputs = ["p1", "p2"] result = [] for elem in inputs: result.append(requests.get(f'http://somesite.com/?p={elem}')) print(result)
This should do what you want.
-
answered 2022-05-04 10:55
mak47
x = ['param1', 'param2'] for z in x r = requests.get('http://somesite.com/?p='.format(x)) print(r)
for z in x
is afor
loop.z could be anything, it's a placeholder for "one object in my array".
Your array has two items
param1
andparam2
so this will do whatever is inside that loop, in this caseprint
them both once.
do you know?
how many words do you know
See also questions close to this topic
-
Python File Tagging System does not retrieve nested dictionaries in dictionary
I am building a file tagging system using Python. The idea is simple. Given a directory of files (and files within subdirectories), I want to filter them out using a filter input and tag those files with a word or a phrase.
If I got the following contents in my current directory:
data/ budget.xls world_building_budget.txt a.txt b.exe hello_world.dat world_builder.spec
and I execute the following command in the shell:
py -3 tag_tool.py -filter=world -tag="World-Building Tool"
My output will be:
These files were tagged with "World-Building Tool": data/ world_building_budget.txt hello_world.dat world_builder.spec
My current output isn't exactly like this but basically, I am converting all files and files within subdirectories into a single dictionary like this:
def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return tree
Right now, my dictionary looks like this:
key:''
.In the following function, I am turning the empty values
''
into empty lists (to hold my tags):def empty_str_to_list(d): for k,v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v)
When I run my entire code, this is my output:
hello_world.dat ['World-Building Tool'] world_builder.spec ['World-Building Tool']
But it does not see
data/world_building_budget.txt
. This is the full dictionary:{'data': {'world_building_budget.txt': []}, 'a.txt': [], 'hello_world.dat': [], 'b.exe': [], 'world_builder.spec': []}
This is my full code:
import os, argparse def fs_tree_to_dict(path_): file_token = '' for root, dirs, files in os.walk(path_): tree = {d: fs_tree_to_dict(os.path.join(root, d)) for d in dirs} tree.update({f: file_token for f in files}) return tree def empty_str_to_list(d): for k, v in d.items(): if v == '': d[k] = [] elif isinstance(v, dict): empty_str_to_list(v) parser = argparse.ArgumentParser(description="Just an example", formatter_class=argparse.ArgumentDefaultsHelpFormatter) parser.add_argument("--filter", action="store", help="keyword to filter files") parser.add_argument("--tag", action="store", help="a tag phrase to attach to a file") parser.add_argument("--get_tagged", action="store", help="retrieve files matching an existing tag") args = parser.parse_args() filter = args.filter tag = args.tag get_tagged = args.get_tagged current_dir = os.getcwd() files_dict = fs_tree_to_dict(current_dir) empty_str_to_list(files_dict) for k, v in files_dict.items(): if filter in k: if v == []: v.append(tag) print(k, v) elif isinstance(v, dict): empty_str_to_list(v) if get_tagged in v: print(k, v)
-
Actaully i am working on a project and in it, it is showing no module name pip_internal plz help me for the same. I am using pycharm(conda interpreter
File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\Scripts\pip.exe\__main__.py", line 4, in <module> File "C:\Users\pjain\AppData\Local\Programs\Python\Python310\lib\site-packages\pip\_internal\__init__.py", line 4, in <module> from pip_internal.utils import _log
I am using pycharm with conda interpreter.
-
Looping the function if the input is not string
I'm new to python (first of all) I have a homework to do a function about checking if an item exists in a dictionary or not.
inventory = {"apple" : 50, "orange" : 50, "pineapple" : 70, "strawberry" : 30} def check_item(): x = input("Enter the fruit's name: ") if not x.isalpha(): print("Error! You need to type the name of the fruit") elif x in inventory: print("Fruit found:", x) print("Inventory available:", inventory[x],"KG") else: print("Fruit not found") check_item()
I want the function to loop again only if the input written is not string. I've tried to type return Under print("Error! You need to type the name of the fruit") but didn't work. Help
-
How to get pass an array through a function with a user input?
I want to ask the user for the size of a 2D array arr[][], but also pass it through the function initializeArray. However, if I pass it through the function, I would have to have a size declarator for col, which doesn't allow the user to enter their own value for the size
#include<iostream> using namespace std; void initializeArray(arr[][10], int N); int main() { int N; cout << "enter an array size: "; cin >> N; int arr[N][N]; initializeArray(arr, N); // I get an error here for(int i = 0; i < N; i++) { for(int j = 0; j < N; j++) cout << arr[i][j] << " "; cout << endl; } } void initializeArray(int arr[][10], int N) { for(int i = 0; i < N; i++) for(int j = 0; j < N; j++) arr[i][j] = 0; }
The only solution I found was the make arr[][] a global array, but in that case, I would have to still declare the size parameters, and I want the user to enter whatever they want. Is there another way to fix this?
-
Two-dimensional array C++
Problem: Write a program of a two dimensional integer array of exactly four rows and four columns, and find the sum of the integers each row of the array as well as determine the smallest inputted element/data in the array. The user will input the elements. Display the elements in tabular form. Make the program user friendly. Use a Class and a member function for the process involve.
-
Pass Associative Array from Symfony Controller to Twig and JavaScript
So, I have 2 arrays that I've combined into one array after making sure they're the same length as such.
//Fetching Ratings for All Coaches & Making a Key, Value Associative Array with Coach id as Key and Rating as Value $coaches = $coachRepository->findAll(); $rating = $coachRepository->findRatingByCoach(); //Array of coach IDs $id_array = array(); foreach ($coaches as $c){ $id_array[] = $c->getId(); } $combined = array_combine($id_array, $rating);
Now I want to pass this key, value array to Twig template and JavaScript. I'm currently doing this
return $this->render('...', [ ..., 'combined'=>$combined ]);
And I'm accessing the array in Twig as such
{% for key, value in combined %} {{ key }} - {{ value }} {% endfor %}
And I'm getting this error.
Object of class App\Entity\Coach could not be converted to string
Looking at the Stack Trace I can see that the array combined is being passed in this form.
'combined' => array(object(Coach))
After a bit of debugging it turns out that I thought the issue was with the $
id_array
when in fact it's the $rating
variable that's presenting an issue. I'm not sure why but I'm getting a single float value. I fixed the issue by changing how I'm fetching the $rating, now instead of using a QueryBuilder to get it from the DB, I'm getting it from list of Coaches as such//Array of coach IDs and array of Ratings $id_array = array(); $rating_array = array(); foreach ($coaches as $c){ $id_array[] = $c->getId(); $rating_array [] = $c->getRating(); }
Even if I set the array
-
Scraping .aspx page with Python yields 404
I'm a web-scraping beginner and am trying to scrape this webpage: https://profiles.doe.mass.edu/statereport/ap.aspx
I'd like to be able to put in some settings at the top (like District, 2020-2021, Computer Science A, Female) and then download the resulting data for those settings.
Here's the code I'm currently using:
import requests from bs4 import BeautifulSoup url = 'https://profiles.doe.mass.edu/statereport/ap.aspx' with requests.Session() as s: s.headers['User-Agent'] = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:100.0) Gecko/20100101 Firefox/100.0" r = s.get('https://profiles.doe.mass.edu/statereport/ap.aspx') soup = BeautifulSoup(r.text,"lxml") data = {i['name']:i.get('value','') for i in soup.select('input[name]')} data["ctl00$ContentPlaceHolder1$ddReportType"]="DISTRICT", data["ctl00$ContentPlaceHolder1$ddYear"]="2021", data["ctl00$ContentPlaceHolder1$ddSubject"]="COMSCA", data["ctl00$ContentPlaceHolder1$ddStudentGroup"]="F", p = s.post(url,data=data)
When I print out
p.text
, I get a page with title'\t404 - Page Not Found\r\n'
and message<h2>We are unable to locate information at: <br /><br ' '/>http://profiles.doe.mass.edu:80/statereport/ap.aspxp?ASP.NET_SessionId=bxfgao54wru50zl5tkmfml00</h2>\r\n'
Here's what
data
looks like before I modify it:{'__EVENTVALIDATION': '/wEdAFXz4796FFICjJ1Xc5ZOd9SwSHUlrrW+2y3gXxnnQf/b23Vhtt4oQyaVxTPpLLu5SKjKYgCipfSrKpW6jkHllWSEpW6/zTHqyc3IGH3Y0p/oA6xdsl0Dt4O8D2I0RxEvXEWFWVOnvCipZArmSoAj/6Nog6zUh+Jhjqd1LNep6GtJczTu236xw2xaJFSzyG+xo1ygDunu7BCYVmh+LuKcW56TG5L0jGOqySgRaEMolHMgR0Wo68k/uWImXPWE+YrUtgDXkgqzsktuw0QHVZv7mSDJ31NaBb64Fs9ARJ5Argo+FxJW/LIaGGeAYoDphL88oao07IP77wrmH6t1R4d88C8ImDHG9DY3sCDemvzhV+wJcnU4a5qVvRziPyzqDWnj3tqRclGoSw0VvVK9w+C3/577Gx5gqF21UsZuYzfP4emcqvJ7ckTiBk7CpZkjUjM6Z9XchlxNjWi1LkzyZ8QMP0MaNCP4CVYJfndopwFzJC7kI3W106YIA/xglzXrSdmq6/MDUCczeqIsmRQGyTOkQFH724RllsbZyHoPHYvoSAJilrMQf6BUERVN4ojysx3fz5qZhZE7DWaJAC882mXz4mEtcevFrLwuVPD7iB2v2mlWoK0S5Chw4WavlmHC+9BRhT36jtBzSPRROlXuc6P9YehFJOmpQXqlVil7C9OylT4Kz5tYzrX9JVWEpeWULgo9Evm+ipJZOKY2YnC41xTK/MbZFxsIxqwHA3IuS10Q5laFojoB+e+FDCqazV9MvcHllsPv2TK3N1oNHA8ODKnEABoLdRgumrTLDF8Lh+k+Y4EROoHhBaO3aMppAI52v3ajRcCFET22jbEm/5+P2TG2dhPhYgtZ8M/e/AoXht29ixVQ1ReO/6bhLIM+i48RTmcl76n1mNjfimB8r3irXQGYIEqCkXlUHZ/SNlRYyx3obJ6E/eljlPveWNidFHOaj+FznOh264qDkMm7fF78WBO2v0x+or1WGijWDdQtRy9WRKXchYxUchmBlYm15YbBfMrIB7+77NJV+M6uIVVnCyiDRGj+oPXcTYxqSUCLrOMQyzYKJeu8/hWD0gOdKeoYUdUUJq4idIk+bLYy76sI/N2aK+aXZo/JPQ+23gTHzIlyi4Io7O6kXaULPs8rfo8hpkH1qXyKb/rP2VJBNWgyp8jOMx9px+m4/e2Iecd86E4eN4Rk6OIiwqGp+dMdgntXu5ruRHb1awPlVmDw92dL1P0b0XxJW7EGfMzyssMDhs1VT6K6iMUTHbuXkNGaEG1dP1h4ktnCwGqDLVutU6UuzT6i4nfqnvFjGK9+7Ze8qWIl8SYyhmvzmgpLjdMuF9CYMQ2Aa79HXLKFACsSSm0dyiU1/ZGyII2Fvga9o+nVV1jZam3LkcAPaXEKwEyJXfN/DA7P4nFAaQ+QP+2bSgrcw+/dw+86OhPyG88qyJwqZODEXE1WB5zSOUywGb1/Xed7wq9WoRs6v8rAK5c/2iH7YLiJ4mUVDo+7WCKrzO5+Hsyah3frMKbheY1acRmSVUzRgCnTx7jvcLGR9Jbt6TredqZaWZBrDFcntdg7EHd7imK5PqjUld3iCVjdyO+yLKUkMKiFD85G3vEferg/Q/TtfVBqeTU0ohP9d+CsKOmV/dxVYWEtBcfa9KiN6j4N8pP7+3iUOhajojZ8jV98kxT0zPZlzkpqI4SwR6Ys8d2RjIi5K+oQul4pL5u+zZvX0lsLP9Jl7FeVTfBvST67T6ohz8dl9gBfmmbwnT23SyuFSUGd6ZGaKE+9kKYmuImW7w3ePs7C70yDWHpIpxP/IJ4GHb36LWto2g3Ld3goCQ4fXPu7C4iTiN6b5WUSlJJsWGF4eQkJue8=', '__VIEWSTATE': '/wEPDwUKLTM0NzY4OTQ4NmRkDwwPzTpuna+yxVhQxpRF4n2+zYKQtotwRPqzuCkRvyU=', '__VIEWSTATEGENERATOR': '2B6F8D71', 'ctl00$ContentPlaceHolder1$btnViewReport': 'View Report', 'ctl00$ContentPlaceHolder1$hfExport': 'ViewReport', 'leftNavId': '11241', 'quickSearchValue': '', 'runQuickSearch': 'Y', 'searchType': 'QUICK', 'searchtext': ''}
Following suggestions from similar questions, I've tried playing around with the parameters, editing
data
in various ways (to emulate the POST request that I see in my browser when I navigate the site myself), and specifying anASP.NET_SessionId
, but to no avail.How can I access the information from this website?
-
scraping NYT mini crossword stats gives 403 forbidden URL error
I'm trying to scrape my NYT mini crossword stats to then update a google sheet. But I'm having trouble with the login portion of the code. Here's my code so far:
from http import cookies import pandas as pd import requests from bs4 import BeautifulSoup import re s = requests.session() nyt_login_url = 'https://myaccount.nytimes.com/' login_route = 'svc/lire_ui/login' auth_token = 'my auth token from requests' HEADERS = { 'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/98.0.4758.109 Safari/537.36', 'origin': nyt_login_url, 'referer': nyt_login_url + login_route } login_payload = { 'login': 'myusername', 'password': 'mypassword', 'authtoken': auth_token } login_req = s.post(nyt_login_url + login_route, headers = HEADERS, data = login_payload) print(login_req.raise_for_status())
When I run this I receive the following error:
requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://myaccount.nytimes.com/svc/lire_ui/login
If I successfully log in, the status code should be 200, but not sure how to get there. Instead, it's 403. Any help?
-
How to convert Curl SSL and auth request to Python code
I would like to convert my request written in Curl to Python. I'm ignoring ssl becouse this certificate is signed by me. I am not sure if I have correctly replaced the value of "description" from null to None. How to improve my query written in Python?
Curl
curl -k -i -u 'admin:admin' --header 'Content-Type: application/json' -d '{"name": "Testowa Rola 221", "description": null, "privilegeIds": [1, 2, 3], "userIds": []}' -X POST https://localhost:44411/api/Role
Python
import requests url = 'https://localhost:44411/api/Role' data = { "name": "Test name 12", "description": None, "privilegeIds": [1, 2, 3], "userIds": [] } headers = { 'Content-Type': 'application/json' } auth = ( 'admin', 'admin' ) verify = '/home/dominik/jpk/xefix/ssl/https.crt' create_administrator = requests.post(url=url, verify=verify, headers=headers, data=data, auth=False) print(create_administrator.text)