We and devs are an Agile team and finally managed to work in parallel on development and testing within an iteration. When developers were working on new features, we were automating test scenarios, preparing test data, implementing fixtures and talking to developers to expose necessary interfaces for those fixtures. So, when developers delivered the features implemented, we already had tests automated to run. That was very encouraging.

However, we found some tests failing because of errors in the tests, like:

  • NullPointerExceptions in fixtures
  • Some configuration required by the tested component was not provided by the test
  • Some small differences in naming convention of expected results (devs called them differently)

Since devs delivered the feature on the last day of the iteration, we didn't have much time to fix our tests and stories were carried over for the next iteration. What are the ways to address this problem?

Proposed solutions

  1. Make devs deliver stories for testing quickier, so we have time to verify our tests as well.
  2. Test our tests before getting the feature from devs. Some problems could be, in fact, avoided with unit testing of fixtures, but others would require mocking the system/feature. That's a lot more work from our side.
  3. We focus on automating integration tests. Those are quite high-level tests, but maybe we are still too tightly coupled with interfaces which the devs expose to us and instead should we focus on less-coupled end-to-end tests?
  • 2
    It sounds like you suffer from the same kinds of pains that developers face during integration testing.
    – user246
    Commented Jun 3, 2013 at 19:19

3 Answers 3


Although it is an old question, I would like to provide my thoughts on this for future readers.I was in similar situation numerous times, in fact, its very common situation for test engineers working in Agile.

Our journey was somewhat as below:

1). We all as a team realised this issue. Dev and test agreed and we planned to move large no. of tests from low/middle level (integration level) to unit level and aimed to have only 1 E2E UI test per user story.

2).Focused to design more black box automated tests without relying on the internals.This is true for any level of tests.

3).However, We made sure(by code review) all the boundary/edge cases are fully covered in unit tests only.Also in unit tests we found we needed to focus more to test public interfaces, not directly the internals.

4).We(testers) worked more closely with devs and helped them in developing/testing/reviewing unit tests.This period we moved more towards TDD(but not exactly). At this point, We also started thinking actually "Are we developers or testers"?

5).After few iterations, we ended up having the majority of tests as unit tests and fewer integration tests and very few E2E tests.

At this point as a team, we found ourselves in a far better situation and were overall satisfied.(Happy Ending :) )

Above all (I think which helped), we had to blur the boundary between who is "Developer" and who is "tester" in the team.

I believe Project Nirvana comes when quality becomes a mindset/culture in the team not just having an individual as "Tester".

  1. You can unit test your own tests as well (we start sounding very meta here - test your tests that test the code) :)

  2. Provide sample input that your tests can use and parse as what should be expected valid/invalid inputs. Confirm the output.

  3. Build tests for the software that expects input. Treat it as test-driven development - the software passes when it passes these regression/acceptance tests that you've constructed.

  4. as devs build or design features, they could give you the component names, or settle on a naming convention that they sign off on. That way they have to stick to it - or they've failed your test for having the wrong component name.

  5. have one iteration with manual tests, and propose automating the previous feature in the next iteration. It means you're one step 'behind' in a way, but it also means you have the option of always having something to test against. Depends what your deadlines are like and how strict your company is with Agile workflow.

The joys of Agile mean you may have trouble fitting this into your environment, as we all do, but it's a case of working with what you have and seeing which best fits your environment / working habits.

  • Re 2: You meant "provide to devs"?
    – dzieciou
    Commented Jun 3, 2013 at 17:21
  • Re 3: We sometimes do so, but I think it does not resolve the problem alone. Passing or failing test won't drive the development correctly if the fault is in the test, not the system under test.
    – dzieciou
    Commented Jun 3, 2013 at 17:22
  • Re 5: I think that would make sense in GUI-based system. We have backend system tests, so automated tests are often easier to perform than manual ones, so in our case that could be a waste of time. But I think the idea here would be to make tests that are more black-box, more decoupled from the code, and so not that much depending on internal details.
    – dzieciou
    Commented Jun 3, 2013 at 17:25
  • @dzieciou - for 2 - no, I mean to your tests - to test your tests, as it were. For 3 - yeah, it's tricky, unfortunately, but it works for some. And at least it'll show a fail - you have to assume, at least initially that your test is correct once you've built it, and then you can investigate once a fail occurs. For 5, backend does make it a bit easier, especially with frequent code drops.
    – Mark Mayo
    Commented Jun 3, 2013 at 23:14

The situation you had given here is really generic and it is in many organizations. For your above problem I can suggest some possible solutions which I had seen.

  • According to your solution 1, you can ask for the developers to do their dev tasks quickly if possible so that maximum testing coverage can be given.

  • In case of new feature which the devs will be adding to your existing system, make sure that after their development you can do : Regression Testing (to check that the new feature will not affect the existing functionality of the system) and also make sure that the new feature is completely tested by your QA team until it is completely stable.

(I know that in Agile, you have much less time for complete testing but that is how the quality can be best achieved without having the stories/tests to be carried over the next iteration)

Rest your approach is perfectly fine of developing the automated tests until the fature is being developed by the devs. That's really awsome and helpful.

The main advantage you may get by having these solutions implemented is completely stable feature can be released and probability of automated test failure will be less.

Second Answer

Let me tell you the workflow for my team in short.

1. Developers (abroad) first develop the feature and then it is ready to test.
2. Then that feature comes to a feature testing team.
3.The feature testing team is completely responsible for manual testing first, and then the development of automated tests.
4. After that the feature testing team will declare that feature as stable.
5. The new feature can then be released and before releasing it may come to Regression Testing team for regression.

In this way, the testing can be done.


  • Can you elaborate on your second point? Particularly, what do you mean by "make sure the new feature is completely tested until it is completely stable"? Do you mean I should start automation only when the feature is developed and stable?
    – dzieciou
    Commented Jun 3, 2013 at 17:28
  • Yes. This is because once the feature is stable, you have the expected behavior and based on that you can develop the automated tests so that the probability of failure is minimal. Commented Jun 4, 2013 at 3:09
  • So what QA should do before devs complete development? My understanding was that QA works in parallel to feature development. Once features gets completed, there's not much time within iteration to automate tests.
    – dzieciou
    Commented Jun 4, 2013 at 4:24
  • I think there is some misunderstanding. You are right, but my point of view above is regarding the development of automated tests. When the feature is being developed by devs QA is involved in Manual Testing. While doing manual testing, QA usually opens new issues those are to be handled by devs during development. Is that answered your question? Commented Jun 4, 2013 at 4:35
  • This answer my question. Thank you. But this approach means I will have even less time to automate tests, as I will be testing the same feature both manually and automatically. How do you handle that in your team?
    – dzieciou
    Commented Jun 6, 2013 at 5:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.