We are doing integration tests where we simulate manually data coming from external systems to our application. Integration tests task has become bottleneck in our team, because preparing test data is hard. The root cause of the problem is in complex format of test data:
- Hard to learn. Developers work on already parsed data, so they don't care about format. But at integration level I must know all the nuances of the format. It takes more to understand the format than to write the test itself.
- Error-prone. In the end you don't know why your test fails. Is it because of the system under test or incorrect test data?
Those are no so big problems for testers that have been involved in the project for long time, but are real obstacles for new-comers like me. So what we're doing:
We're creating test data generators. We're encoding into the generator the knowledge from documentation and answers of business analysts, other testers, and developers. Test data generators contain pre-defined blocks of data that a tester can compose together to create complete test data. A tester must learn DSL (based on Java), obviously, but I hope it is easier than the final format.
We're creating test data validators. Test data created by a generator are validated against some basic constrains (e.g. mandatory fields, values correct across different part of test data) before they get serialized to a specific XML format. That eliminates basic mistakes.
What else can we do to simplify the process of test data creation?
How can we make sure test data are correct?