Downloading Website in Python

How would I go about downloading web pages using Python in such a way that I could view them offline as well as sift through their html code for information?

Just like right clicking and clicking Save-As on any web page.

1 answer

  • answered 2018-03-13 21:47 Nenad

    You should definitely look into Requests and Beautiful Soup.

    The simples example would go something like this (note: both of these libraries you have to install first using pip python -m pip install <library>):

    import requests
    from bs4 import BeatifulSoup
    res = requests.get('')
    soup = BeautifulSoup(res.text, 'html.parser')

    The last line will print out the entire html of the website but it will also format it in a readable form just like when you look at the html by inspecting element on a website.

    If you want to download the page and put it in a separate html file you can do that like this:

    with open('example.html', 'w', encoding = 'utf8') as webSite: