1

With huge sets of integration tests, data setup is becoming somewhat of an obstacle in terms of runtime.

What is your experience to tackle similar situations?

Scenario: 200 API integration tests concerning contracts, for which data prerequisites include the existence of a company and a person, each in a specific (non-default) state.

Case 1: each test sets up its own data

  • Pro: test data is managed within test, no interference with other data
  • Con: creating new companies/people for every test greatly slows down total runtime

Case 2: data setup is done mostly on test project level

  • Con: test data is managed across all tests in the project, making it harder to manage and prevent cross-influences
  • Pro: creating reusable companies/people the data setup is much smaller, leading to way faster runtimes (tried and tested...)

Case 3: data setup is done using database restore/snapshots

  • Con: test data is managed across all tests in the project, making it harder to manage and prevent cross-influences
  • Pro: creating reusable companies/people the data setup is much smaller, leading to way faster runtimes
  • Con: running and debugging individual tests against a deployed environment will be very difficult and time-consuming if the entire database needs to be restored
  • Please clarify the following: You seek faster execution of your test cases or less time to maintain the test cases? Clarify what "project level" means(how many "projects" you have)? How is you automation structured and if necessary share the language that you use? – chesthar Sep 3 '19 at 20:12
  • @chesthar I realise my question can be easily split in two separate questions. I'll be editing this one soon. – FDM Sep 12 '19 at 7:38
  • @chesthar done! – FDM Sep 17 '19 at 8:02
1

If multiple tests can use the same datasets (from your question it seems they can), then I think Case 3 is more efficient, because you will spend less time overall creating the test datasets (not loading, but creating (ie, writing the SQL scripts)). You will create it once and use it for many tests.

It will be a "Pro" (not a "Con") to manage it at project level: you will have the visibility of all test cases and what they all require, so you can see patterns and similarities, and assign the same test dataset to test cases with similar test data requirements.

You should, of course, have some document that says which dataset is used by which tests. And when writing your test cases, you should have a "test data requirements" section where you list what kind of data is required for the test (example: "have 2 companies in the DB, one with 4 people and one with 5 people").

You can also start loading the datasets before your test execution begins (if that is possible on your environment and budget). That will save you some "test preparation" time during test execution. You can also use docker containers (like the other answer said) to store these datasets+DBs to speed up your load time.

0

Mix of 1 and 5. Similar to the use-case at Zoosk - database with data for separate tests or whole test-suite residing in docker containers to improve load / import times.

  • Rephrased my question to be more concrete/focused – FDM Sep 17 '19 at 8:08

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.