There seem to be three predominant setups for running automated tests.

  1. The company invested in a selenium grid that is always available and the nightly tests run against them.
  2. The CI will build a test runner, selenium grid on demand, run the tests and then wind down after the job is finished
  3. Have the test runner, tests, browser driver and browser in an image, build the container on demand, run the tests and then wind down (not much popular but the fastest in my opinion)

Till recently I was a fan of the third option. I used to run ruby based selenium tests in a container built from an image containing ruby, chrome driver and chrome. The tests are stable, runs fast. These used to run nightly on Jenkins.

Now, I am writing a new test framework using javascript and webdriverio. Trying to use browserstack/sauce labs with job runners like GitLab pipelines or bitbucket pipelines seems to be very slow. And we are not keen of setting up our own permanent selenium grid.

I am curious to know what do the industry follow and what is the best suggested approach is

  • 3
    There are no best practices, only good practices in context. Please add more focus on your question, by adding more about your context (product and people) and what problem are you trying to solve (it seems to be performance, but "seems to be very slow" is not precise). – João Farias Apr 30 '20 at 5:53
  • If you don't need dedicated selenium grid then your 3rd approach is the best in this case also, what stops you from adapting that ? – PDHide Apr 30 '20 at 8:42
  • My company wants to stay away from maintaining own servers in the view that it is easier to use a cloud provider for selenium grid. But, like I said the test runs are very slow. For comparison, a set of tests running sequentially that run in 3 minutes on local take 15 to 20 minutes when pointed to browserstack. I loved the third approach but, I do not see many images on docker hub that come with selenium, programming language, browser driver and browser built-in. This make me feel that this is not the right way to do it. If it is, there should be docker images, conversations about it online – rails4sandeep May 1 '20 at 0:33
  • Did you try running the tests in parallel on browserstack/saucelabs ? Trying to run tests parallelly at a test fixture level gives a boost to your run time and it is cheaper – VSD May 4 '20 at 20:39

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.