How do I decide what the exact xpath is to find and open all profiles on a web page using Selenium and Python?

I am new to programming, and I am trying to build a test software to automate the data retrieval from a website. I am, however, finding it difficult to define the xpath, so that all of the individual profiles can be found and clicked on.

'/html/body/div[1]/div[2]/div/div[3]/div/main/table[2]/tbody/tr/td/div/table/tbody/**tr[2]/td[1]**/div/table/tbody/tr/td[1]/a'
'/html/body/div[1]/div[2]/div/div[3]/div/main/table[2]/tbody/tr/td/div/table/tbody/**tr[2]/td[2]**/div/table/tbody/tr/td[1]/a'
'/html/body/div[1]/div[2]/div/div[3]/div/main/table[2]/tbody/tr/td/div/table/tbody/**tr[3]/td[1]**/div/table/tbody/tr/td[1]/a'

I am finding it difficult, because the variables are in the middle of the xpath and don't know how to write the profilesCount and the for loop. Sorry, if my question sounds dumb, I have read all resources I could think of, but found no solution. Thank you for your help in advance.

1 answer

  • answered 2018-11-08 12:14 Dominik Lašo

    tell me if I am wrong

    if I got it right, you want to make a loop thats going to check all profiles on website .... If its the same problem as I had in my proxy_gatherer, then this solution may help you:

    for x in range(2,34):
        try:
            u1 = browser.find_element_by_xpath('//*[@id="tbl_proxy_list"]/tbody[1]/tr[' + str(x) + ']/td[1]/abbr').text
            except:
                print('proxy out of range')
    

    in your case ( based on all informations you provided ):

    for x in range(2,last_profile_id):
        for xx in range(1,2):
            browser.find_element_by_xpath('/html/body/div[1]/div[2]/div/div[3]/div/main/table[2]/tbody/tr/td/div/table/tbody/**tr[{}]/td[{}]**/div/table/tbody/tr/td[1]/a'.format(x,xx))
    

    explanation what i did: based on xpaths you provided I analysed which parts are changing ... here it was :

    // tr[2]/td[1] // -> its written bold here :

    '/html/body/div[1]/div[2]/div/div[3]/div/main/table[2]/tbody/tr/td/div/table/tbody/tr[2]/td[1]/div/table/tbody/tr/td[1]/a'

    I hope I helped you :) if no and I completely missunderstood what did you ask, please let me know ^^