Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

We currently have a two week sprint and we seem have issues with timing between testing and development. We have (nearly) daily releases to a test environment. We stage our release about 1.5 days before the production release in a separate environment. We have 3 developers and 1 tester.

One strategy is to test through these daily releases as features/bug fixes are code-completed. The drawback with this strategy is that the code base is in constant flux and what passed testing at the beginning may not pass at the end. There is a little bit of time to retest things during the 1.5 day window, but not enough to test everything.

The other strategy is to stagger the testing, so that the tester is working on the (frozen) code base one sprint behind the developers. The drawback with this is that if problems are found, the functionality needs to be remembered/rehashed/etc - a lot of the arguments against waterfall.

Does anyone have any opinions about which way is better, or an alternative?

Thanks,

share|improve this question
1  
"what passed testing at the beginning may not pass at the end" - why is this, aren't there automated tests that are picking up these failures quickly ? –  Phil Kirkham Aug 29 '12 at 16:08
2  
can the devs help the tester with the testing if there really is 'too much' ? shouldn't the whole team be helping rather than having the devs carry on creating work that the tester cannot test ? –  Phil Kirkham Aug 29 '12 at 16:10
    
That's a good point, Phil, but if automated tests could pick up everything, they wouldn't need a manual tester too. And in many corporate cultures, slowing the devs down is -never- a valid option. You and I both agree that untested software is dangerous and risky, but many (too many) places consider thorough testing a "nice to have". –  corsiKa Aug 29 '12 at 17:53
    
Phil, we currently working toward more automation, but we aren't there yet. You bring a valuable point that this would help alleviate the issue we are experiencing. –  user2101 Aug 29 '12 at 19:07

5 Answers 5

up vote 5 down vote accepted

If you want to go fast, you need to assume that once something is tested and working in a cycle, it will continue to work in that cycle. If you cannot make that assumption, you either need to spend more time testing (by yourself or with the help of others) or your developers need to deliver higher-quality code. No one but you and your developers can decide how to deliver higher-quality code; the path you take depends on your circumstances.

For the first few years, my company also had three developers and one tester. Now we have four developers and two testers, but we still take a two-phase approach to testing: a feature test, where we focus on things that we know have changed, and then a regression test, where we test across the whole product without regard to what has changed. The feature test begins as soon as a feature is code-complete.

The regression test begins when most/all features are complete and feature-tested. It is a safety net for catching bugs in areas that are presumably unrelated to what has changed. Whether you need a separate regression test will depend on how your product is structured and how much it changes from one sprint to the next.

share|improve this answer

No matter what you decide, staggering code and test sprints is ALWAYS a bad idea. You double your delivery lead time (at least), reduce collaboration, increase management complexity, lose velocity stability, and incur more bugs.

share|improve this answer

This is a very common situation in agile teams. In order to tackle it we have a big set of automated tests with different granularity written by devs and testers. However that is not enough. A different developers' mindset is required in order to achieve an early deliver for partially completed, and easy to test, features.

This is, just imagine a developer is programming a windows service that runs periodically and can be configured via a UI. That dev. should deliver to QA the windows service as soon as possible with an easy way to verify its behavior and continue with the UI.

Basically, testers cannot wait a lot to test new features and devs cannot wait for testers. Then, deliver small functionalities helps everyone to have something in what work on.

Another thing is that even when requirements can change, devs have to try to don´t change the code but work around it.

share|improve this answer

In my humble opinion, there are two things that are key to solving this: You have to automate most of your testing, and you have to use a strict test-first approach. Doing this you will be able to specify the tests first and run them immediately and automatically for each release. You'll catch any regressions that way, too.

Of course, this is not easy to implement at all. Some things are way harder to automate than others. And also, automating tests first (especially UI tests), depends on a mature test and development infrastructure. One hint would be to use data-driven and keyword-driven testing (and definitely not any kind of record/replay).

There will always be a need for some manual testing, even if just for exploratory testing, but the first step towards testing in agile projects will always be to automate as much as you can, and as soon as you can.

share|improve this answer

This is the exact problem that bedevils the environment where I work, and have yet to find a strategy that works well and consistently.

Some of the strategies and techniques that help are:

  • The testing specialist works with the coding specialists on the unit tests. Even if the testing specialist isn't a coder, knowing the coverage of the unit tests and discussing them with the coders allows a more effective use of time across all the team.
  • Following code completion, coding specialists perform manual tests. Discussion during unit test development helps people who don't have much experience with testing to perform happy path testing.
  • During development and exploratory testing, the testing specialist identifies core tests to perform in the last day and a half, as well as a small number of test sequences designed to flush out likely problematic areas. These get performed first, while code specialists are performing the acceptance and happy path tests. It helps a lot if the team is genuinely agile rather than "scrum-fall" or "agile-fall".
  • Test automation, whether API or GUI-based runs a sprint behind so that the automation specialist is working on a stable code base.

Frankly, in my experience there is no best way - only a set of techniques and strategies you can use to minimize problems.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.