How to integrate 4 QA engineers into 4 dev (Scrum) teams?

Question

In the Product Team I work in, the engineering resources look as follows:

• 4 dev teams (4-5 members each), each team with a dedicated Product Owner
• 1 QA team (4 members)

Approach 1:

Currently, the single QA team works together in order to manually test tickets coming from all 4 dev teams. This approach is working but it also creates questions about priorities: which ticket of which team is more important to test? how do the dev teams choose a tester to assign the tickets to? etc.

Approach 2:

What we are thinking of, though, is assigning 1 QA engineer to each dev team. This, of course, raises the question of truck factor, as having only one dedicated QA engineer to work with a single dev team seems risky and might be inefficient.

Question:

From your experience, what would be the most optimal way to integrate the QA team into our product development? Are there any reasonable alternatives to the 2 approaches above?

Answer 1

You have Scrum in your tags, so the first point worth noting is that Scrum requires cross-functional teams. I assume QA is required to deliver product to production, so that skill needs to be in the teams.

Now, with a 4:1 ratio there is a very real risk of them being overwhelmed, especially if all of the work is handed to them on the last day of the sprint. Here are a few things to consider in this regard:

1) Nothing says a team member from another team can't help if needed. This is a band-aid to the problem, but sometimes a band-aid is exactly what you need.

2) There are many ways to run development and testing in parallel. I highly advise looking at TDD, BDD, and Spec-by-Example for ideas of how to not wait until the end of the sprint to test. (I know this doesn't cover all types of testing, but it really helps spread the work over the whole sprint)

3) The value of a good QA Engineer is not in running tests or documenting tests. It is in identifying what to test - the cracks and angles the developers don't think about. That means that a lot of the labor of testing can picked up by any scrum team member.

4) One of things you definitely lose in this model is people with like skillsets helping each other grow. A simple "chapter" or "community of practice" model that lets testers meet together once a week to share ideas and best practices helps mitigate this.

I've worked in both of these models as both a developer and a tester. Option 2 always got me better results.

Answer 2

From a QA perspective:

Usually, when a tester is assigned in a team, there are few phases of testing that he/she goes through. The most important ones are:

1. Dumb Monkey Testing:

   • Use the system as a completely new user who has '0 knowledge of the product'

   • Try out the system in a way it is not meant to be used.

   • This allows in identifying the robustness of the system, security issues, handling errors, recovery time etc.

2. Exploratory Testing:

   • Use the system as a completely new user who has '0 knowledge of the product'

   • Use it as a user who is trying to learn the product.

   • Use it as a user who tries to understand error messages, tooltips, button design etc. to understand the correct way of using a product.

   • This allows in identifying the flaws in the UX design, identifying usability, accessibility-related issues etc.

3. Functional Testing:

   • Use the system as a domain expert who knows the domain and product.

   • Use the system in a way who knows the business requirement.

   • Test highly complex business use cases etc.

So coming to your question:

The advantage of the first approach is that:

This will give you more results in Dumb monkey testing and exploratory testing. As the testers domain knowledge and product knowledge would be limited in scenarios where they test multiple products than concentrating on a single product. ( Because we all are humans and will get distracted when multi-tasked).

Disadvantage of first approach:

The testers will lack domain expertise and product expertise that could lead to missing out testing complex business scenarios and bugs being pushed to production.

The advantage of the second approach is that:

The user gets more time in improving knowledge about the product and domain.

This will give more results in functional tests as the tester is well aware of complex use cases of the product and domain. This allows making sure the product works the way it intended to.

Disadvantage of second approach:

As the tester is well aware of the product and domain, sometimes they miss in identifying UX flaws that might prevent a user from being attracted to buying the product.

So exploratory testing and monkey testing would be less efficient here (But a well-skilled tester might even cover this part)

So my tip:

1. The First approach is better for products that are building from scratch and are not mature enough.
2. The second approach is best for all scenarios and can find complex scenarios in early testing itself. (Given you don't overload the tester)
3. Hire more QA if a single person can't handle it alone.

Answer 3

My recommendation is to take a different approach.

I suggest splitting QA into two functions:

1. Exploratory testing using traditional approaches to try and break stuff. This would remain as a separate group

2. Automated testing written by qe engineers who are involved in development of both the application and the automated tests for it and are embedded on the application teams.

The main challenge with this solution is that you'll need to hire more staff. Maybe not twice as much though. Maybe 2 can do exploratory testing. Remember exploratory testing is NOT^ exhaustive and using all data combinations and devices - it is more of a sampling approach.

As others note, having 1 qa/qe on a team... that needs to write UI testing... that is for completed features... by 3 developers... with little time left in the sprint... leads to this just not working.

Answer 4

The second approach you have shared makes more sense to me. In that way, each QA resource will have better understanding of a single product/module rather than know everything coming from all four dev teams, hence , better outcome without wastage of time on multiple domain modules. The QA team may often sit together to discuss the overall domain if necessary.

Answer 5

Approach 1 is just waterfall as you hand over work to another discipline. I dislike it because it introduces context switching as defects found ping-pong between the QA and the original DEV team.

For approach 2, consider starting with a single team and if it works scale to the other teams.

having only one dedicated QA engineer to work with a single dev team seems risky and might be inefficient.

It reduces risks as other team members should be able to cover the work of the single point of failure. The dedicated QA is not the do-guy but the goto-guy to help and coach other team-members to deliver better quality. Consider pair-testing and three-amigo-sessions.

I would consider investing in a Agile Testing Fellowship course for the whole team:

a three-day training course for agile teams and testers. Participants learn ways the whole software delivery team can collaborate to plan and execute testing activities needed to build quality into their product.

Suggested patterns:

• Swarming: One-Piece Continuous Flow
• Three amigos