1

I am executing a script which clicks through a number of pages to retrieve data. After each click the new page obviously has to load and the time to load varies greatly (mostly 2-3 seconds, sometimes > 20 seconds). Thus, I need to make the code flexible to wait with executing until the element is loaded. I have implemented the solution below, but I still get a stale element reference error, which I do not understand (as the code should not exit the while loop before the loading is completed). Can anyone explain why this happens/ what I can do to resolve?

Sorry in advance, I am not able to share an executable script with you.

error = 1 
        if page_ref < Max_pagenum_1: 
            while error == 1: 
                try: 
                  link = 
                    browser.find_element_by_xpath('//li[contains(@class, 
                    "page-item") and 
                    contains(@class,"active")]/following- 
                    sibling::li/a').click()
                    error = 0
                    time.sleep(3)
                except: 
                   error = 1
                    time.sleep(20)
        else: print('finished' + Name) 

Max_pagenum_1 is the last page for each item page_ref is the currently active page

1
  • Can you provide the stack trace provided with the error? Your code does not seem suspicious.. Stale Element exception is usually thrown when you bind the element on the page to the object in your code and then that saved element is no longer attached to dom.
    – Alexey R.
    Oct 16, 2018 at 13:22

2 Answers 2

2

You should wait when xpath is visible before clicking like that

from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC

WebDriverWait(driver, 15).until(EC.visibility_of_element_located((By.XPATH, '//li[contains(@class,"page-item") and contains(@class,"active")]/following-sibling::li/a'))

It means, wait until xpath is visible (max 15 second), then you can continue to click.

1

This is a classic problem with using Xpath and then calling a click method instantly.

before running the click method on it you need to find the element, then pause , check its still active, then run the click method on it.

you should also avoid having a catch all exception handler, instead specify the exceptions you want to catch. If you aren't specific you program will be confusing when it goes wrong as it conflate all errors.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.