I'm using Selenium Webdriver for testing a Django/Python based web application. I use both, the phantomjs and firefox drivers. We use django's LiveServerTest test case, which basically starts up a test webserver for the app.

Now, on our CI environment (wercker), I sometimes - but not always - get a TimeoutException from my webdriver.get(url) calls. The thing is, though, that when I look at the screenshots (which I collect on exceptions) the actual page seems to be rendered just fine.

I'm already using a technique to wait for a page reload that is described here: http://www.obeythetestinggoat.com/how-to-get-selenium-to-wait-for-page-load-after-a-click.html. This wrapper basically waits until the old page's HTML element is replaced with the new one. It seemed to resolve similar problems for when I run my tests locally.

So, what am I experiencing here? How can I proceed? Any hints greatly apreciated.

Edit: I followed Artem's advice and remove all external resources (fonts and google maps calls). I initially thought that helped but then got other errors instead. I now get TimeoutExceptions in a click() call... Here's the log:

Traceback (most recent call last):
  File "/pipeline/build/flatfox/tests/ui/utils.py", line 57, in wrapper
    func(self, *args, **kwargs)
  File "/pipeline/build/flatfox/tests/ui/test_auction_cycle.py", line 157, in test_manage_applications
    self.driver.find_element_by_css_selector('.modal-dialog .btn[name=reject]').click()
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 65, in click
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webelement.py", line 385, in _execute
    return self._parent.execute(command, params)
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/webdriver.py", line 173, in execute
  File "/usr/local/lib/python2.7/dist-packages/selenium/webdriver/remote/errorhandler.py", line 166, in check_response
    raise exception_class(message, screen, stacktrace)
TimeoutException: Message: u'{"errorMessage":"Click failed: timeout","request":{"headers":{"Accept":"application/json","Accept-Encoding":"identity","Connection":"close","Content-Length":"81","Content-Type":"application/json;charset=UTF-8","Host":"","User-Agent":"Python-urllib/2.7"},"httpVersion":"1.1","method":"POST","post":"{\\"sessionId\\": \\"d67c58a0-610d-11e4-a75e-eb735e81fd4c\\", \\"id\\": \\":wdc:1414767284194\\"}","url":"/click","urlParsed":{"anchor":"","query":"","file":"click","directory":"/","path":"/click","relative":"/click","port":"","host":"","password":"","user":"","userInfo":"","authority":"","protocol":"","source":"/click","queryKey":{},"chunks":["click"]},"urlOriginal":"/session/d67c58a0-610d-11e4-a75e-eb735e81fd4c/element/%3Awdc%3A1414767284194/click"}}' ; Screenshot: available via screen 

Edit: Here's the code making the click():

with wait_for_page_load(self.driver):
    self.driver.find_element_by_css_selector('.modal-dialog .btn[name=reject]').click()


def wait_for(condition_function):
    start_time = time.time()
    while time.time() < start_time + 10:
        if condition_function():
            return True
    raise Exception('Timeout waiting for {0}'.format(condition_function.__name__))

class wait_for_page_load(object):
    def __init__(self, browser):
        self.browser = browser

    def __enter__(self):
        self.old_page = self.browser.find_element_by_tag_name('html')

    def page_has_loaded(self):
        new_page = self.browser.find_element_by_tag_name('html')
        return new_page.id != self.old_page.id

    def __exit__(self, *_):
        except Exception, e:
            logging.getLogger("test.ui").error("wait_for_page_load(): failed with %s", e)

I'm a bit lost now and wonder whether it has something to do with the CI environment...

Thanks, Bernhard

  • What locators are you using for waiting? I assume you use only explicit waits and expected_contition, am I right? +1 for good researched question. Show us the code. – Peter M. - stands for Monica Oct 30 '14 at 17:12

In selenium the wait command doesn't really wait sometimes. That is there is some problem with that command.

You can try the time.sleep(); command of python. It is an equivalent of the Thread.sleep(); command of Java.

It will pause the execution of code for defined time limit. So you can give sufficient time that will allow your page to load.

Hope it works... :)

| improve this answer | |
  • Ah, I forgot to mention it, but I've already surrounded the calls by time.sleep(1)s. This did indeed help on my local environment, but doesn't solve the problem on wercker. – Nuschk Oct 31 '14 at 7:37
  • 2
    Did you try it with more time? Maybe 5 seconds instead of 1 or maybe 10. It may be that the time u defined is not sufficient. – IAmMilinPatel Oct 31 '14 at 7:41
  • I have my wait settings at 8 for both, page reload and implicit wait... I'm now trying to up those numbers... Thanks – Nuschk Nov 1 '14 at 10:45
  • 1
    Okay. Try that and do let me know if it works. A lot of people are having trouble with the wait command of selenium and in java usually the Thread.sleep() seems to resolve it. So i'd like to know if the python equivalent does the trick as well! – IAmMilinPatel Nov 1 '14 at 10:48
  • Hmm, okay. I didn't surround the click command with sleeps, so I'm going to try that one as well... – Nuschk Nov 1 '14 at 10:50
wait = WebDriverWait(driver, 10) 
element = wait.until(EC.element_to_be_clickable((By.ID,'someid'))) 

Try above code and see if it is working or not.

| improve this answer | |
  • Yes, I've tried that. The thing is, I don't really get out of the get() call at all, since it already triggers the Exception. – Nuschk Oct 31 '14 at 7:39
driver = webdriver.Firefox() 
driver.implicitly_wait(10) # seconds 
driver.get("http:// enter your URL.") 
myDynamicElement = driver.find_element_by_id("myDynamicElement") 

Try above code and see if it is working or not.

| improve this answer | |

The possible reason might be that page looks to be rendered (UI elements are in place), but not completely loaded in browser.

If there is any background calls, especially to 3rd-party services, it might take a while to reach them and get response, only then browser is able to mark the page as finally loaded.

| improve this answer | |
  • Ah, nice, I'm going to look into that. I do have a few external resources on my page (fonts, maps). I try to disable those and see if that helps. – Nuschk Oct 31 '14 at 7:41

thank you all for your suggestions. Though it's still not perfect, I now have a better working environment. I combined Artem's suggestion of removing external dependencies with Pushpendra's tip to boost up implicit_wait and page_load_timeout further up. They are now at 12 and 30, but I intend to go even higher. I've also increased the sleep times, as suggested by milinpatel17, to as high as 4 seconds around calls that don't reload the page, but modify the DOM in a way.

The core of the problem seems to be that the CI environment's cpu loading sometimes is unpredictable and I have to make enough room for those times it is laggy. This is also what wercker's support suggested (nice guys, btw). I will now make these timeout values dependant on the environment, such that I can run local tests fast and then slow down on the CI.

I can't mark a single answer as valid, and, unfortunately, my reputation doesn't allow upvoting... sorry. But many thanks for your support!

| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.