Python Selenium Web Scrape embedded csv/excel file in XPATH to pandas dataframe

I am working on a Python requirement to download data from a secured website using Python Selenium WebDriver. I am using xpath.click() to download the file to a local download folder or any specific folder I wanted to.

Instead of downloading to a folder, I wanted to load to a database table.

I am looking for help to know how I can direct the downloaded file to a table.

data = driver.find_element_by_xpath('//*[@id="modules--reports-gridTar-instanceGrid"]/div/div[1]/div/div[2]/div[2]/a').click()
print(data) // output: None

The downloaded file is seen on the web browser:

Not sure how could I pick that file showing on the browser to a table?

1 answer

  • answered 2018-11-08 08:20 ewwink

    don't click() but get href attribute to download with python.

    data = driver.find_element_by_xpath('//*[@id="modules--reports-gridTar-instanceGrid"]/div/div[1]/div/div[2]/div[2]/a')
    url = data.get_attribute('href')
    req = urllib2.urlopen(url).read()
    response = req.read()
    print(response)