Test isolation and automation

Question

We are using Selenium web services for our automated functional testing.

I have a test that tests the creation of some objects. If I run the same test again, I need to make sure that each test has a different business key - else it will run into a unique-constraint violation.

One way, I can fix this is to reset the database before\after every run of test suite so that each test suite doesn't conflict with another. However, my individual tests within the same run are not isolated from each other.

How do I get to the point, where all my tests are isolated and do not conflict with each other?

Right now we query the database to generate random IDs that do not exist in the database. But I am sure there must be a more elegant way of solving this.

Answer 1

RN.

You've got several options here. One is, as you mentioned, to reset the database before each run. That's probably a good idea anyway to ensure that you've got the same starting condition and the beginning of the run.

For the individual tests within the run, you can operate a simple counter through the run and use that to generate your business key - that is, your list of objects includes an object number in its data, and your key is generated using that object number.

Some possible options are a timestamp to milliseconds (e.g. 201112081110526) which you save for the test to allow you to check the saved object against your input. An iterator of 1, 2, 3... etc will also give you the same result.

Using something like this will give you a set of tests that aren't going to conflict with each other. Unless you have tests that depend on another object having been created in an earlier test, they should also be independent of each other. Whether your end-of-run database will match your baselines if one test fails will depend on how intelligent your baseline checking is - if you check each object individually after creation, one failed test won't impact baselines, where if all the object checks occur at the end of all creation tests, one failed test could have a major impact. Either way, you'll still have a cleaner test set with known unique IDs that you don't have to run a SQL query to generate.

Answer 2

One possibility is to reset the database before every test, not only before every run of the suite.

This will likely make each run take longer. If you can tolerate the increased duration, that's they way I'd go.

And it may require adding setup to tests that now depend on earlier tests to establish their initial conditions. But that's another way of saying that it makes the tests independent, which is your stated goal.