Selenium WebDriver sometimes fails to connect to the browser when running tests in headless mode

Question

I'm experiencing timeouts when Webdriver connects to the Chrome browser in headless acceptance tests setup on Linux.

I'm using the following setup to run the tests:

• Vagrant machine with Ubuntu 14.04
• Running tests in headless mode using xvfb
• Selenium, ChromeDriver 2.20.353124, Google Chrome 46.0.2490.80
• Capybara as a layer on top of Selenium, RSpec for asserts, Spinach as a main test runner + some other gems (I can provide the full list if required)
• Ruby 2.1.5
• The tests are being run either by executing the Jenkins job (which just logins via SSH to remote server and executes the script which spins up vagrant box and runs the tests) or manually by starting vagrant box on local machine and running the tests inside it
• Application under test is on the remote server

The command to run the tests looks like this:

xvfb-run --server-args="-screen 0 1024x768x24" rake ci:chrome:en

The last one is the name of rake job which calls an executable within ci_reporter_spinach Gem (required to generate reports for Jenkins) which in turn calls Spinach tests runner itself.

The tests I'm running are using multiple instances of Chrome browser, utilising $capybara_session_manager for that.

The issue I have is happening very inconsistently, about 10% of the tests runs fail at the initial prerequisites steps when I'm initializing a new session for the user and open new browsers. That's how it looks when the debugging is enabled:

D, [2015-10-27T16:18:09.769591 #25406] DEBUG -- : [httplog] Connecting:    127.0.0.1:9516
D, [2015-10-27T16:18:09.770319 #25406] DEBUG -- : [httplog] Sending:    POST http://127.0.0.1:9516/session
D, [2015-10-27T16:18:09.770437 #25406] DEBUG -- : [httplog] Data:   {"desiredCapabilities":  {"browserName":"chrome","version":"","platform":"ANY","javascriptEnabled":true,"cssSelectorsEnabled":true,"takesScreenshot":false,"nativeEvents":false,"rotatable":false,"chromeOptions":{"detach":true},"chrome.detach":true}}

here test stops some time and then fails with error:

 Net::ReadTimeout
    /home/vagrant/.rvm/rubies/ruby-2.1.5/lib/ruby/2.1.0/net/protocol.rb:158:in `rescue in rbuf_fill'

because the browser fails to respond in 60 seconds.

If I enable the debug mode that returns me pry session on any test fail, I'm able to initiate the session creation once again by calling:

pry(main)> $capybara_session_manager.for_back_office.driver.browser
D, [2015-10-27T16:26:44.938492 #25406] DEBUG -- : [httplog]  Connecting: 127.0.0.1:9517
D, [2015-10-27T16:26:44.939049 #25406] DEBUG -- : [httplog] Sending: POST http://127.0.0.1:9517/session
D, [2015-10-27T16:26:44.939254 #25406] DEBUG -- : [httplog] Data: {"desiredCapabilities":{"browserName":"chrome","version":"","platform":"ANY","javascriptEnabled":true,"cssSelectorsEnabled":true,"takesScreenshot":false,"nativeEvents":false,"rotatable":false,"chromeOptions":{"detach":true},"chrome.detach":true}}
D, [2015-10-27T16:26:45.320160 #25406] DEBUG -- : [httplog] Status: 200
D, [2015-10-27T16:26:45.320328 #25406] DEBUG -- : [httplog] Benchmark: 0.380709069 seconds
D, [2015-10-27T16:26:45.320417 #25406] DEBUG -- : [httplog] Response:
{"sessionId":"cbb6d5d5bb994f804ef46aecf3ea3a3b","status":0,
  "value":{"acceptSslCerts":true,"applicationCacheEnabled":false,"browserConnectionEnabled":false,"browserName":"chrome",
    "chrome":{"userDataDir":"/tmp/.com.google.Chrome.QYfMR8"},"cssSelectorsEnabled":true,"databaseEnabled":false,"handlesAlerts":true,"hasTouchScreen":false,"javascriptEnabled":true,"locationContextEnabled":true,"mobileEmulationEnabled":false,"nativeEvents":true,"platform":"Linux","rotatable":false,"takesHeapSnapshot":true,"takesScreenshot":true,"version":"46.0.2490.80","webStorageEnabled":true}}

I was not able to find any good solution for this one in google. One important point - I was not able to reproduce the error when running tests locally on OS X machine in non-headless (i.e. with real browser) mode.

Also, I"m pretty much sure that the error has nothing to do with the tests itself or the application under tests, it is something related only to communication of Webdriver and Chrome browser and xvfb is also affecting this somehow.

Answer 1

I was able to determine what was causing the issue. After new session initialization:

session = Capybara::Session.new(driver)

I was calling following method to maximise the browser window:

session.driver.browser.manage.window.maximize

Looks like sometimes a new browser window is opening too slow and when Capybara tries to maximise the window, this request times out because browser wasn't opened yet.

One of the possible solutions is to check that browser window really responds:

try_a_few_times { session.driver.browser.title.include? 'data:,' }

where try_a_few_times is a name of method that does the catch/retry:

  def try_a_few_times(how_many=5, tries=0, &block)
    begin
      yield
    rescue Exception => exception
      raise exception if tries > how_many
      puts exception if Environment.debug?
      sleep 1
      try_a_few_times how_many, tries + 1, &block
    end
  end

I still have no idea why this was happening only when I run tests in headless (xvfb) mode but now I was able to get rid of this error.