We and devs are an Agile team and finally managed to work in parallel on development and testing within an iteration. When developers were working on new features, we were automating test scenarios, preparing test data, implementing fixtures and talking to developers to expose necessary interfaces for those fixtures. So, when developers delivered the features implemented, we already had tests automated to run. That was very encouraging.
However, we found some tests failing because of errors in the tests, like:
- NullPointerExceptions in fixtures
- Some configuration required by the tested component was not provided by the test
- Some small differences in naming convention of expected results (devs called them differently)
Since devs delivered the feature on the last day of the iteration, we didn't have much time to fix our tests and stories were carried over for the next iteration. What are the ways to address this problem?
- Make devs deliver stories for testing quickier, so we have time to verify our tests as well.
- Test our tests before getting the feature from devs. Some problems could be, in fact, avoided with unit testing of fixtures, but others would require mocking the system/feature. That's a lot more work from our side.
- We focus on automating integration tests. Those are quite high-level tests, but maybe we are still too tightly coupled with interfaces which the devs expose to us and instead should we focus on less-coupled end-to-end tests?