What are the signs of an understaffed QA team?

Question

What are the typical signs and signals of a Quality Assurance team being understaffed? How would you justify/explain a request for a new addition to a team?

---

A little bit of background:

We currently have few full and part time testers but with growing demands and products we've built a rather long backlog of test stories and scenarios to be automated. Our test automation efforts are sometimes one or even two releases behind the new functionality, features and fixes. I am not absolutely sure I can use the above mentioned facts to justify a request for a new test automation specialist since our management is not really fully aware of the scale of our test automation and what testers actually do on a daily basis - there is a bit of a disconnect between QA team and product management.

Answer 1

I will answer this question with Agile iterations in mind. (An iteration could be a full Sprint or a single-user-story-cycle if you do Kanban or swarming.)

The QA department is understaffed when:

• Not every cross-functional Agile team has a someone with QA knowledge
• The ratio defects versus released features is getting out of control
• Test automation is not completed during the iteration
• Manual testing is executed in the next iteration
• Working software is not shipped at the end of iteration (This could have other causes as well.)
• Un-tested software is shipped
• QA employees have no time for a QA-community which focuses on growing and learning from each other during work hours

we've built a rather long backlog of test stories and scenarios to be automated

I have learned what helps best is to visualize it. Somewhere prominent in your company show you are behind constantly. Just make a QA metrics area in the main hallway. Post things as test-automation cycle-time and lead-time, number of stories still to automate, defects vs new features. Preferable add some targets you would like to achieve with a trend arrow per day/week/month.

I am slightly worried by your words "long backlog", this clearly shows the work is not done. Accumulating work not done should be alarming to management. I would push the development teams to add testing (including test automation) to their definition of done. Adding more test-engineers will probably not solve your problems anyways.

Experiment with scaling test-engineers by making them part of the development teams, let the team figure out what their perfect ratio devs versus testers should be. This will be different per team. But integrating testers in the team has a lot of benefits. Whole team quality attitude, better knowledge sharing, no-handovers and really DONE!DONE! (e.g. production ready) to name a few.

Answer 2

When they are not able to prove the value they add

In my last position I wanted to improve front end error messages for invalid data in html web forms. My pleas fell on deaf ears. The company is focused on profit and costs. Then I used AB testing to show that better error reporting could earn the company millions of dollars a year. Next request for more resources was immediately approved. Show you add value and you won't need to try and persuade by argument, the facts speak for themselves.

Above is the big picture. The smaller signs of an understaffed qa staff are:

• shortcuts taken when writing tests
• not enough time to develop mock and stub strategies
• intermittent failures that no-one has the time to fix
• qa being perceived as a bottleneck
• event and time based tests that are tied to a real clock so take hours or days
• insufficient time to develop good practices
• insufficient time to evangelize quality practices
• insufficient time to make the case for good testing environments
• long running and slow tests that no-one has time to refactor

In terms of making the case to management I recommend:

• constant evangelism about why testing matters
• frequent references to things like healthcare.gov (performance testing), target.com (security), Amazon (accessibility testing)
• use humor to make a point about integrated testing, there are lots of examples
• Make to to keep telling management that you are in the job of preventing errors

Answer 3

An understaffed QA department is one where a tester is not able to effectively test the work of those that are producing surface area (code, content, documentation) for her to test.

The ratio of producers to testers can be used as an initial rule of thumb for how close to properly staffed you are. The worst case scenario I've had in my career was 17 producers to 1 QA resource back when I worked on video games. The best case was around 4 producers to 1 QA resource when I worked on a mobile computer vision product.

If testers are so focused on having to test what has already been produced to the point where they can't spend any time thinking about how to test what is coming, it's a no-win scenario. They won't be able to get through the backlog quickly enough to provide feedback close enough to when bugs are introduced to allow bug fixes and design changes to happen in a cost-effective manner, and by not being able to address code before it's written through design reviews, they won't be able to stop bugs from coming into the project in the first place.

Answer 4

Our company provides wide range of software testing services which includes mobile testing, functional testing, automation testing and many more. We analyze the requirements and plan for the resources accordingly. Various factors contribute to make proper resource plans such as project estimates, release cycles, scope of testing i.e web functional,mobile devices, automation.

Below are the signs of below strength QA team.

1. Declining quality of product If QA team is understaffed, Project Quality takes a hit. Bugs starts to leak which are encountered by customers post release. This leads to frequent hot fixes and high rates of unsatisfactory customers.

2. Big Release Cycles. This is another issue which is faced by understaffed QA team. Timelines extend with short staffed QA Team. For Example if full regression testing for a release is needed in certain period, short staffed QA team will always push the release deadline.

3. More tasks in Backlog. The tasks begins to compile up in backlog. Most ominous of all, automation takes a backseat which in turns makes things worse in terms of release cycles.