Timing of development and testing in agile environment

Question

We currently have a two-week sprint and we seem to have issues with timing between testing and development. We have (nearly) daily releases to a test environment. We stage our release about 1.5 days before the production release in a separate environment. We have 3 developers and 1 tester.

One strategy is to test through these daily releases as features/bug fixes are code-completed. The drawback with this strategy is that the code base is in constant flux and what passed testing at the beginning may not pass at the end. There is a little bit of time to retest things during the 1.5-day window, but not enough to test everything.

The other strategy is to stagger the testing so that the tester is working on the (frozen) codebase one sprint behind the developers. The drawback with this is that if problems are found, the functionality needs to be remembered/rehashed/etc - a lot of the arguments against the waterfall.

Does anyone have any opinions about which way is better or an alternative?

Thanks,

Answer 1

If you want to go fast, you need to assume that once something is tested and working in a cycle, it will continue to work in that cycle. If you cannot make that assumption, you either need to spend more time testing (by yourself or with the help of others) or your developers need to deliver higher-quality code. No one but you and your developers can decide how to deliver higher-quality code; the path you take depends on your circumstances.

For the first few years, my company also had three developers and one tester. Now we have four developers and two testers, but we still take a two-phase approach to test: a feature test, where we focus on things that we know have changed, and then a regression test, where we test across the whole product without regard to what has changed. The feature test begins as soon as a feature is code-complete.

The regression test begins when most/all features are complete and feature-tested. It is a safety net for catching bugs in areas that are presumably unrelated to what has changed. Whether you need a separate regression test will depend on how your product is structured and how much it changes from one sprint to the next.

Answer 2

This is the exact problem that bedevils the environment where I work, and have yet to find a strategy that works well and consistently.

Some of the strategies and techniques that help are:

• The testing specialist works with the coding specialists on the unit tests. Even if the testing specialist isn't a coder, knowing the coverage of the unit tests and discussing them with the coders allows more effective use of time across all the team.
• Following code completion, coding specialists perform manual tests. Discussion during unit test development helps people who don't have much experience with testing to perform happy path testing.
• During development and exploratory testing, the testing specialist identifies core tests to perform in the last day and a half, as well as a small number of test sequences designed to flush out likely problematic areas. These get performed first, while code specialists are performing the acceptance and happy path tests. It helps a lot if the team is genuinely agile rather than "scrum-fall" or "agile-fall".
• Test automation, whether API or GUI-based runs a sprint behind so that the automation specialist is working on a stable codebase.

Frankly, in my experience, there is no best way - only a set of techniques and strategies you can use to minimize problems.

Answer 3

In my humble opinion, there are two things that are key to solving this: You have to automate most of your testing, and you have to use a strict test-first approach. Doing this you will be able to specify the tests first and run them immediately and automatically for each release. You'll catch any regressions that way, too.

Of course, this is not easy to implement at all. Some things are way harder to automate than others. And also, automating tests first (especially UI tests), depends on a mature test and development infrastructure. One hint would be to use data-driven and keyword-driven testing (and definitely not any kind of record/replay).

There will always be a need for some manual testing, even if just for exploratory testing, but the first step towards testing in agile projects will always be to automate as much as you can, and as soon as you can.

Answer 4

This is a very common situation in agile teams. In order to tackle it, we have a big set of automated tests with different granularity written by devs and testers. However, that is not enough. A different developers' mindset is required in order to achieve an early deliver for partially completed, and easy to test, features.

This is, just imagine a developer is programming a windows service that runs periodically and can be configured via a UI. That dev. should deliver to QA the windows service as soon as possible with an easy way to verify its behaviour and continue with the UI.

Basically, testers cannot wait a lot to test new features and devs cannot wait for testers. Then, deliver small functionalities helps everyone to have something in what works on.

Another thing is that even when requirements can change, devs have to try to don´t change the code but work around it.

Answer 5

No matter what you decide, staggering code and test sprints is ALWAYS a bad idea. You double your delivery lead time (at least), reduce collaboration, increase management complexity, lose velocity stability, and incur more bugs.

Answer 6

First, determine if the type of development that you need to do lends itself to agile well. There are many simple development processes that fit well within agile, and many deep dive development processes that are a nightmare to try to use agile for. Second, if you are working on something that is simple to implement and test in less than one sprint, you should be able to mix the tester in when the design is being worked on with the customer, the tester can then write up the tests and setup the data while to coder codes, and then testing goes fast.

Of the 5 agile jobs I have had, most are just using the word agile to hide wild wild west. Only one worked well with testing and the customer.