The test team as an enemy of development? And how can this be avoided?

Question

Details

Forming a Scrum team should include all the skills necessary to develop a user story in order to deliver a potentially deliverable product increment with each sprint.

In traditional organizations, however, I always encounter a fundamental mistrust of the integration of testers in the Scrum teams. Instead, a separate test team is to remain, which is then responsible for regression tests, load and performance tests and the test automation. The rationale for this type of organization is the so-called independence of the testers.

I have several problems with this view of things. Scrum makes the team fully responsible for the results. The establishment of an "independent" test team assumes that the Scrum team does not live up to its responsibilities and would turn a blind eye to errors in the product increment.

Another danger associated with the independent test team is that the testers become test report reporters who are not involved in the elimination of the problem.

In the scrum sense, we prefer problem solvers. The tester in the Scrum team, as well as all developers responsible for the delivery of a flawless product increment and will make every effort to fix it or have it fixed when uncovering an error. Another advantage of the tester in the team is the simple possibility to develop automated tests in step with the implementation of the user stories.

The Problem:

the procedure described already shows a part of the problem: the lead time for a new Product Backlog Item increases to several Sprints: 1 Sprint implementation plus 1 Sprint deferred test (plus possibly another Sprint error correction, if unfortunately this is no longer possible, without the commitment to break the current sprint, and considered less important). This results in further problems: does one need 2 Definition of Done? When does the PO take the item? Does he take it off twice? How much buffer does the deployment team need to keep in fix to fix the returned bugs? Not to mention the context switch that becomes necessary. Pull testers and developers together and try to avoid mistakes instead of finding (with 1 sprint offset)? etc etc

How to change this problem?

Answer 1

I've tested in the sprint + 1 system under the SAFe framework. The framework does not specific this but lends itself to doing it for organizations coming from waterfall.

Mu suggestion is:

stop it

Your questions of

Does one need 2 Definition of Done? When does the PO take the item? Does he take it off twice? How much buffer does the deployment team need to keep in fix to fix the returned bugs? Not to mention the context switch that becomes necessary. Pull testers and developers together and try to avoid mistakes instead of finding (with 1 sprint offset)? etc etc

plus ones that I would add such as:

How to keep the code bases branched correctly? How to keep code in sync with environments? How to manage deployment through multiple environments and tests and processes? How to record the bugs?

When I find I am writing the words above I pause and go back to:

• Individuals and interactions over processes and tools
• Working software over comprehensive documentation
• Customer collaboration over contract negotiation
• Responding to change over following a plan

particularly

• Individuals and interactions over processes and tools

Testing is sprint+1 is introducing a whole lot of process instead of individuals doing the work now and talking to each other. This sort of set up will inevitably lead to "The test team as an enemy of development" and that is what you have found. Exactly that

You need to keep stressing the importance of changing this. It is an investment. It will slow development down this week... and speed it up in X months. Leadership for the long term view is needed and can come come from any self-empowered person in the organization.

If you cannot change the setup I recommend the following actions:

• Write failing tests first (BDD)
• Pay equitably for automation engineers
• Communicate the benefit of testing to developers
• Work on relationships between application and automation engineers
• Embed automation engineers within the application development teams
• Truly empower automation engineers to 'pull the cord' and say no, don't deploy
• Talk openly about second class citizen syndrome for testers and how to avoid it
• Ensure social events - lunches, parties, lunch and learns, etc. include both parties
• Refer to folks as application and automation engineers instead of 'devs and testers'

Answer 2

Get a good Scrum Master who can convince the organization that Scrum teams should not be depended on other teams to deliver shippable software. It is an impediment he/she should resolve.

Traditional Organisations want the benefits of Scrum without changing their ways. Even for great coaches, this could be a process of years. Don't give up. Be bluntly honest about these ScrumBut Mini Waterfalls (eg testing after the Sprint) to management. Good Scrum leadership should work on fixing it. I think your thinking is spot on, but trust has still to be earned. See if you can find one team who dares to help prove that your thinking works. Maybe ask dev and test team for 2-3 Sprints to Experiment with your ideas.

The rationale for this type of organization is the so-called independence of the testers.

The counter-argument is that having an independent test team is that development teams can take shortcuts to make their Sprints because the testers will find their mistakes. Leading to dev-test ping-pong and lower quality because the test team is also under pressure to release and will skip low risks tests over high-risk area's. Leading to a slower release cycle and overall lower quality.

Scrum Testers should create a quality culture in the Scrum team, coaching team members to understand how to produce a well-tested increment at the end of the Sprint.

Load and performance testing could be a separate Product Backlog Item to improve performance. Although automating this type of testing in build pipelines is becoming more and more common.

update:

Alan Page did convince me having a separate security testing specialist team might be very important depending on your context and risks.

Answer 3

The cynic in me feels that if developers could be relied on to detect & fix their bugs as they go along then separate testing teams would never have been invented in the first place.

Much zealotry in this thread...

Answer 4

Trust your feelings, Luke. Seriously, the scenario you described is an anti-pattern. They simply are not ready to let go of waterfall. They are probably concerned about the panic among test leads and managers and testers themselves when they realize that their services as they currently exist are no longer needed. But this is really a binary thing. You either test continuously (all dev team members) and be agile or you hand it off and remain waterfall. Which do they want?

Answer 5

If you cannot finish a PBI in one sprint, it is too big and needs to be split up into smaller parts.

Finishing a PBI means it has to be shippable and conform to your DoD, which normally means it has to be tested.

There is absolutely no reason why a team couldn't build and test in the same sprint. If you have a good DoR, a tester can start preparing tests as the code is being built, and executing those tests can be done in a matter of minutes - with plenty of time left to fix any found defects.

Regression tests, load tests, integration tests are maintained in the same sprint as where an increment is delivered.

If you cannot finish coding on time to allow for testing within the sprint, your PBI is simply too big and the team should consider thinking about smaller increments.

A separate testing team should not be able to find actual defects: why did your team deliver something that did not conform to specifications - that seems to be the most essential part of your DoD?

If new specifications pop up during post-sprint testing, they are new wishes and can be handled as any other change request. If actual deviations from original specs is found, your PBI should not have been considered finished. In order to check if your team's work matches the requirements, you need to... test! Which means that even with a separate test team, you still need testers in your own scrum team.

Answer 6

If you cannot have your way, try to go for a compromise that satisfies both stakeholders, you (and perhaps your team*) and the corporation management.

I actually often find a mix rather helpful, where the team has some internal QA people that do in-development testing, but that also coordinate with external testers for final release testing or production bug report verification etc.

That way you can have your cake and eat it too.

• Your team is responsible and can use in-team QA expertise to help the developers track down bugs,
• but it uses existing company resources to stem bigger tasks.
• And that way release/production testing can be set up in a way that it also forces you to be able to hand your product/new features over to external partners ensuring your documentation is up to date. Basically making sure you provide proper interface documentation for whatever you build and having people with an independent mind-set re-check your product.

Btw. you should get it out of your head that any other party being involved with your product would automatically take away your ownership of or responsibility for your product. You don't do security pen-testing in-team or consider pen testers your enemy either (I hope!). Having external testing (on top of internal testing) is just the acknowledgement that you might not see everything as you are so entrenched in your product and how it is supposed to work, you might overlook problems that someone just looking at the surface might rather find. Whether that quality level is required, how it is best implemented (automatic vs. manual etc), are details, but in principle you shouldn't feel somehow attacked because corporate wants to (also) have external testers take a look at your product.

*I'd be careful with assuming your vision for how the team is supposed to work in scrum is supported by the team without clarifying with them.

Answer 7

How about making one member of the programming team work as tester (or quality improver) for a month or two, and then rotate them out for another member of the team (as mentioned in another comment already)? Unlike 'normal' testers, they will not only understand better what to look for (on a code level, for instance), but also be able to help automate the testing - requiring debug and coding standards, employing automated testing which catches errors during software development, and so on.

The normal testers continue as usual and can use some of the tools developed by the programmers for their work. They will also be needed to evaluate how effective the internal testing was, i. e., whether their changes are kept or discarded and whether they get longer or shorter terms or more or less power to change things. This is the real kicker, as it will ensure that only those unpopular changes are pushed through which actually show positive results. While someone with good scores can even get rid of old procedures to ensure quality, if they think it is not necessary anymore with the new measures - thus keeping the process streamlined.

Once your quality goes up, a member of the testing team can rotate in every second turn. By then, people will be used to the interference, and this may add more ideas, which, depending how easy they are to set up, can be done directly or by one of the next programmers testing things.