Our functional test suite is slow
I have inherited a functional test suite using Selenium 2 (Webdriver) and Junit. We have around 500 tests written for our Java web application. Running the full suite takes several hours, so we only run it overnight on our CI server. I want to speed up our test suite by running our tests in parallel, I have researched various solutions for parallel execution including using Selenium Grid.
How we currently handle data
Our test suite currently drops and restores the database in between each test class, we do this to have a known state and avoid data conflicts between tests. However, if we were to run several tests in parallel restoring the database would likely results in errors unless it was coordinated across tests (I wouldn't want to drop the database while another test is in the middle of its execution).
Some possible solutions:
Here are some ideas:
- Create test groups (ex: tests which can run concurrently vs tests that have to run sequentially)
- Have each test remove the data it creates (not always possible)
- Run each tests using a different user to minimize data conflict (very time consuming to create 500 users)
- Deploy multiple instances of our web application, each with its own database (each test would have to known which instance its running on and drop the correct database)
Edit: These guys touch on this issue http://youtu.be/oX-0Mt5zju0?t=9m10s