Coping with an undersized QA team

Question

We're building two inhouse products:

1- The first has web, android, and iOS (total of 9 developers) versions and has 2 QA engineers (1 full-time manual/automation engineer, the other is a 50% dedicated manual QA engineer).

2- The other product has a web version (3 developers) with 1 full-time manual/automation QA engineer.

Our QA team for project #1 is clearly outsized by the development team's productivity of releases. This is delaying getting valuable feedback on our releases.

Although we're trying as hard as possible to hire new QA resources, this is proving challenging and time-consuming. We have deadlines looming by (product #1 in 1 month, product #2 in 2 months) and we need fast and effective ways to handle this shortage in QA.

Are there any tips you can provide for such scenarios?

Answer 1

I don't think your team is ridiculously understaffed. We run our operation more lean than you do.

In real life, ratio of developers to integration/system/functional automation testers is as low as 10:1.

How is possible to maintain quality? By focusing more on unit tests (written and maintained by developers).

Unit tests should be your first line of defense. Because if unit test breaks, it is pretty obvious what broke and how to fix it. Many projects have more lines of code in unit tests than in the production code. And use coverage analysis to make sure that unit test exercise most of the core code.

With such comprehensive battery of unit tests, automated UI/functional/system level test need to cover only 20% of code which covers 80% of common functionality (to keep reasonable quality with least amount of automation). Automated regression tests are code, and needs to be maintained when tested code changes, so you want to have least amount of the regression tests (so you have less test to maintain).

It seems to me that problem with your projects is not the low number of QA testers, but insufficient unit testing and low unit test coverage.

See also debate about test pyramid and Pareto principle about the need to focus.

Answer 2

Risk based testing can help you with quick feedback

This is another good aspect that you should look into.

As stated in one of the answers, developers should be asked to test their work. This is definitely going to help. I completely agree with that.

But, in addition to that "Risk Based" testing can help you further.

• In risk based testing, you try to ensure that most common application flows are tested.
• This way, you might miss some of the "rarest" scenarios but at least you will have a list of most common workflows that are not working.
• So, you pick "High" and "Medium" priority test cases and test them. You leave low priority cases. If the time crunch is too much then you might end up testing “High" priority test cases only; for the purpose of initial feedback.
• But, that doesn't mean that the testing is complete. But, that does mean that commonly used application areas are either "working fine" or "have issues".

Once you have recruited QA resources and your team is NOT out of balance anymore, you can cover what was missed earlier because of lack of time.

But, this gives you a fair idea (if not complete) of project health. And developers will get important defects to work on.

Answer 3

As it seems to me, the ration of dev-to-test in project#1 is obviously overoptimistic. One dev might change one line of code that would require huge test efforts.

However once you faced such the situation (looks like there were some missing risk assessment on project start if you have such the ratio) I would recommend to force your devs to test (give them a chance to demonstrate their productivity one more time) unless you have enough QA resources hired. After all this is what Scrum declares as a value - team cross-functionality.

Answer 4

I quite agree with PeterMasiar and Aalok. Good unit testing can nip some code issues in the bud and priority-based approach may help reduce testing to a reasonable optimum. There are also some measures a tester can take to optimize regression testing, a largely disliked but necessary type of testing. I believe in your case there are 2 viable options:

1. Two-level approach to regression testing. You just divide regression testing into iteration regression and full regression. Iteration regression concentrates on features and changes made in the iteration and areas of the application that could be affected.Full regression covers the whole application and is usually performed before major releases.
2. Risk-based approach Aalok offered can also be extended to regression testing.Testers concentrate on high and medium priority regression test cases. This way they fully cover critical system functionality and exceptional conditions (negative test cases, boundary value test cases, etc.).

Going back to overall testing, exploratory testing may also help detect bugs saving time. But here a tester only relies on his or her experience and 'gut feeling'.

Answer 5

The team size doesn't seem to be the main problem here. If you are currently recruiting with deadlines approaching within 1-2 months then this was a management issue. This should have been brought up earlier so you have the time to employ more staff, as it will still take any new tester a few months to get use to a new system to be of any use, but I digress.

Since you are already in this position what I would do is try and shift the mindset. Although its easy to throw testing at testers, it should be the responsibility of everyone to ensure quality software.

• If you have a deadline in 1 month, does this mean all the necessary features have been implemented and the code is now, or close to, being froze? At this point - as others have suggested - the developers can now help with testing.
• If you are still adding new features and are using agile, this is where you start dropping less important work to ensure a decent minimum viable product can be released that still satisfies the customers requirements.

Adding a new tester now will likely not help with the short term goals as you will have to spend time to hire, teach them the new system, etc.

So, my advice would be to try and make sure everyone is focused on quality. Try and get developers, managers, anyone within the company who can use the software to use it, focusing on as much coverage as possible. Good luck.

As a side note: If you have time, I suggest How Google Tests Software as it has some interesting insights into their strategy for testing very quickly and, more importantly, ensuring a quality product is delivered without having thousands of testers.