Acceptance Testing & Scrum - How do you handle bugs that occur during acceptance testing in parallel to ongoing/upcoming sprints?

Question

Details In the ideal Scrum world, no errors occur in shippable code that has been successfully unit-tested; that this is no longer the case in reality, at the latest after a first manual acceptance test, is, I think, more in line with all of our experience.

My question now is: How do you handle bugs that occur during acceptance testing in parallel to ongoing/upcoming sprints? Do you generally create new user stories out of bugs, or do you only track them separately in a bug tracking system?

In the case of new user stories, I find estimating them somewhat problematic, as the effort required to fix bugs is, in my opinion, more difficult to estimate than that required to implement new stories.

Moreover, yes, an ongoing sprint should remain untouched, but what about serious bugs that more or less prevent a meaningful continuation of acceptance testing?

Henrik Kniberg suggests in his book "Scrum and XP form the Trenches" to basically continue with the implementation of new stories, but to prioritize reported bugs. This approach also makes sense to me, but I am still not completely clear about the optimal (organizational) handling of bugs.

Answer 1

Henrik Kniberg is generally right. When a defect is found after the work is considered "Done" and is in some downstream activity, like UAT, put any issues found on the Product Backlog and order them based on when the team should be looking at them. The team can refine them and start work on them when it makes sense to. In the event of a critical issue, the Product Owner can even bring it to the Developers in the middle of a Sprint to determine what it would take to have it get attention immediately, rather than going through refinement and waiting for the next Sprint Planning session.

For critical issues that are brought into the Sprint immediately, the team is doing this at some level of risk. There is a lot of uncertainty since the team didn't have time to refine the work and get a deeper understanding of the defect and intended behavior before starting, so this will have to be done just-in-time. For other issues, the team can use refinement to gain a deeper understanding of how to reproduce the issue and ensure that environments are set up to properly test solutions.

I would strongly advise against separate systems, or any kind of isolation within a system, between user stories and bugs. Both represent work that needs to be done by the team to improve the system. Through a Scrum lens, there is only a Product Backlog and Product Backlog Items, with no mandatory labeling of what type the item is. If Product Backlog Items are given types, it should be done to benefit the team.

I'd also recommend considering applying root cause analysis techniques to defects found in user acceptance testing, as well as in production. Minimally, spending some time to talk about them at retrospectives would be warranted. These issues represent things that have escaped all of the activities that the team does to understand the work along with all of the quality activities that support the design, development, and Definition of Done associated with a Product Backlog Item. If defects are getting past the team, there may be opportunities to improve the process or at least watch for trends that may suggest a process improvement.

Answer 2

My opinion in this would be create separate defect for it and link it with relevant story and add it in your test suite or cases so that you do not miss it again or make sure that does not occur again. This is what we follow as well, generally. It is natural that bugs would occur in Acceptance test phase.