Prepare data for groups of test cases

Question

I am going to have a family of test cases that are going to work on a single data set. This data set needs to be loaded into a database as an initial step which is shared between all the test cases.

What is the best way to do so? Define a precondition with the insertion of the data?

Answer 1

Ideally I would like tests to be independently executable, so that you can repeat a failing testcase without running the whole test-suite. So it should be a precondition for each test to load the dataset, now you could add a unless it was previously loaded clause.

If it is an automated test you could do the same. Check if the data is available, if not load the data.

If loading data is fast, create a simple button/script to clean&load the data, now you know each test run has a clean set of data, preventing testcases to become depend on each other.

Answer 2

There are a few variables here:

• Will the tests modify the data? - If they will, you'll need to reload the data as a precondition for each test.
• How long does it take to load the data? - If it takes a long time to load the data (and what constitutes a "long time" depends on your situation) you will likely want to reuse the data
• Does the data impact other tests? - If you have other tests to run that will be impacted by this data, you're going to need clean-up to remove it once you're done, and you won't be able to add it until the first of your tests that use it.
• Can the tests with this data be isolated? - If you can isolate the tests using the data, it may be worth your time to isolate them and run them separately to reduce the likelihood of negative impacts to your other tests.
• Are the tests manual or automated? - For automated tests, a method that loads your data and gets called at the start of the test and a teardown method that unloads it could be your best option. For manual tests, things may be more time-consuming.

For context

At my previous employer, we had a lengthy, complicated setup our automated tests needed to use. We dealt with this by running a once-a-week setup script (it took about 4 hours to run), and all our other automation started by restoring the database and local files generated by the backup script. That way, we had the prerequisites we needed and our overnight runs took a lot less time than they would if every script had to do its own setup.

Since we had hundreds of potential configuration flags to deal with, this method worked for us, and the dependency on the database restore rarely caused issues. We knew it was a dependency and that we'd get an essentially worthless result if the restore failed, but it was still better than trying to run the setup (minimum 1 hour) for each script.

There's no real "correct" answer - only what works best for you in your situation.