How can QA regression test automation increase software development output?

Question

I was asked to join a new project where the key objectives are:

• Increase throughput and increase the quality of testing – will contribute to faster delivery of working software i.e. Agile WoW.
• Automate testing in order to increase throughput in the software development life cycle (SDLC).

If the development of a big project takes the development team 24 months to deliver for testing, and the QA test team takes 4 weeks to sign off, how can it possibly help the development team in delivering more software if the 4 week testing process is improved?

I am trying to figure out a way to motivate why our focus should rather be improving the efficiency of the development team because it would be easier to reduce the 24 months, than it would be to reduce the 4 weeks in QA.

Answer 1

Question: how can it possibly help the development team in delivering more software if the 4 week testing process is improved?

Answer: It can't.

Details:

...key objectives are: - Increase throughput and increase the quality of testing – will contribute to faster delivery of working software i.e. Agile WoW.
- Automate testing in order to increase throughput in the software development life cycle (SDLC).
If the development of a big project takes the development team 24 months to deliver for testing...

As other point out this is NOT agile development. You reference Agile here but also say that this is waterfall.

It's pretty simple though:

• If waterfall and 24 months to deliver to QA then manual testing is the correct and appropriate technique to use. There isn't enough time for comprehensive automated tests that would add value and also the tests are written after development is over.
• If actually agile, tests (including both unit and feature tests) will be written throughout those 24 months and the last 4 weeks is looking for unusual edge cases, performance issues, device/browser issues that haven't been detected to date. 95%+ of the tests should be written during the 24 months.

In the second case QA regression test automation, once written, will:

• reduce the time for QA to test basic functionality manually
• free up QA to dive into more complex cases that require manual testing
• increase the speed of development as developers can run tests while developing*
• reduce breakages in current functionality which derail and slow development*

* these are the primary 'increase development output' points.

Please note that many efforts to 'increase software development output' are counter-productive. It is better to take a whole test, whole system approach to development software with quality embedded throughout the proces. I suggest you read Agile Testing by Lisa Crispin and Janet Gregory for ideas, it will give you a lot of them.

Finally I recommend you focus on four factors:

• make sure developers write comprehensive unit tests during development
• make sure developers use TDD/BDD so that tests drive application code development
• make sure developers and QA folks write feature tests during development
• get QA to contribute to planning and test plans to add value before any code is written

Focus on doing those things well and you're going in the right direction.

Answer 2

usually adding automation can-

improve test coverage, although it might suffer from the Pesticide Paradox testing the same paths again and again.

related to the above it will free up testers time to do more un-scripted exploratory testing.

bring quality closer to the developers and provide faster feedback loops, especially if you work in an iterative way (e.g. Agile, Scrum or just short-ish cycles).

greatly improve scalability, you can test more versions, more products etc.

On the other hand test automation doesn't come free, many times you need to invest in test automation the same effort as in the product itself including ongoing maintenance and the Pesticide Paradox, or just not being able to cover enough, will annoy your stakeholders since they will still see bugs being discovered after testing.

To answer your question about shortening development- you probably can't, but you will be able to improve quality

Answer 3

I would assume that with agile process, QA team collaborates with developers during whole 24 months of development, not only during last 4 weeks. I also assume developers have comprehensive unit tests - if not, all hope is lost.

How such collaboration can improve output?

• QA can read requirements (as second set of eyes) and design rests to make sure requirements are met.
• QA will develop automated tests for partially finished features, and test them as they are completed. This (and unit tests) allows developers to refactor rest of the system as user's understanding of the system (and requirements) change during the development.
• Failed tests allows developers to identify suspected code faster 9most recent additions are most suspect).
• So when final version is developed in 24 months, most of the system was comprehensively tested, and only most recent additions are vulnerable.

Also, for most companies, developing first version of the system is just that - first version. It IS very expensive.

Does your company plan to continue developing next version (and you better have good regression tests), or your plans is to deliver the source code to someone and forget about it? If so, any tests above the requirements are waste of time, obviously.

Savings from testing are in making following development and refactoring faster and cheaper. If you "code and forget", your approach might be very different: hire a Perl wizard and pray. :-)

Answer 4

One of the ways automated regression can improve developer throughput is in catching regression bugs faster - something that's critical to the success of waterfall projects (It's critical to agile projects too, but agile projects usually have a lot less time between development of a feature and testing of that feature).

Once you have a reasonably comprehensive automated regression suite running against the latest build at least once a day, you can guarantee that any regression problems introduced by developers will be caught within a day of the change being integrated to the main code branch. That means that the developer's work is still relatively fresh, which in turn means it will be easier for the developer who introduced the problem to correct it.

That fast turnaround imposes a much lower cost in research and analysis on the entire team, as well as freeing the test team to focus more on testing the new code and exploring its integration and interactions, and less on tedious, error-prone regression checks.