We are doing integration tests where we simulate manually data coming from external systems to our application. Our integration tests have become the bottleneck in our process, because preparing test data is hard due to the complex format of test data:
- Hard to learn. Developers work on already parsed data, so they don't care about the format. But at an integration level I must know all the nuances of the format. It takes more to understand the format than to write the test itself.
- Error-prone. In the end you don't know why your test fails. Is it because of the system under test or incorrect test data?
Those are not significant problems for testers that have been involved in the project for a long time, but they are significant obstacles for new folks like myself. So what we're doing is:
We're creating test data generators. We're encoding into the generator the knowledge from documentation and answers of business analysts, other testers, and developers. Test data generators contain pre-defined blocks of data that a tester can compose together to create complete test data. A tester must learn DSL (based on Java), obviously, but I hope it is easier than the final format.
We're creating test data validators. Test data created by a generator are validated against some basic constrains (e.g. mandatory fields, values correct across different part of test data) before they get serialized to a specific XML format. That eliminates basic mistakes.
What else can we do to simplify the process of test data creation?
How can we make sure that test data is correct?