Handle errors that occur in Acceptance Testing, in parallel with running and upcoming sprints?

Question

In the ideal scrum world, shippable code, which is successfully unit-tested, does not fail; That in reality, at least after a first manual acceptance test is no longer that way, I think rather corresponds to our experience. My question is: How do you handle errors that occur in Acceptance Testing, in parallel with running / upcoming sprints?

Do you generally create new user stories from bugs, or do you track them separately in a bugtracking system?

In the case of new user stories, I find their estimation somewhat problematic, since the effort to troubleshoot is, in my opinion, more difficult to estimate than those for implementing new stories.

In addition, a running sprint should remain untouched, but what about serious mistakes that more or less prevent a meaningful continuation of acceptance testing?

Answer 1

It depends

Every team works out its own process for handling bugs that escape to acceptance testing (or beyond).

The method I prefer works this way:

• Any bug found during acceptance testing or later is triaged and prioritized.
• Bugs are treated at the same level as user stories, and both are tracked in the same system.
• If the bug is complex, it becomes a research bug where the estimate is for the level of effort required to work out the core problem and a good solution. A follow-up bug with the same priority is created for the fix.
• If the bug is simple, the estimate is for the fix.
• Each sprint a number of the highest priority bugs will be added along with user stories.

Since I work in an organization where there is only a single team, this method suits our process. Larger organizations with multiple teams might devote a team to bug fixing only, or shift which team is responsible for what depending on the highest priorities within the organization.

The best thing you can do is to monitor your process and look for ways to improve it, then get buy-in from your team about the improvements you're suggesting. That way your process will keep evolving to meet your team, organization, and application needs.

Answer 2

Q: How do you handle errors that occur in Acceptance Testing, in parallel with running / upcoming sprints?

Ans: Note the first agile value which says "individuals and interactions over processes and tools". So there is no need to waiting for the story to get completely build to do an Acceptance testing. I can provide a tip from my experience, check with developer when ever possible on how the feature looks like, give QA feedback right away, also check with the product owner as often as you can. So the feature can get more and more refined while developing and there wont be much surprises during the Acceptance testing.

Q: Do you generally create new user stories from bugs, or do you track them separately in a bug tracking system?

Ans: There is no hard and fast rule for this. You can capture the bug during the testing, get it prioritized right away with the product owner. If the bug fix takes only seconds don't even bother to raise a bug. Quality matters over process. For bugs which takes longer to fix, do raise it as a bug and do the estimation and commitment based on teams capacity.

Q: what about serious mistakes that more or less prevent a meaningful continuation of acceptance testing?

Ans: There is no point in delivering something which the user doesn't want. So the priority should be on the deliverable rather than the scrum process. Think about a Minimum Viable Product and try to deliver it as soon as possible to the user. Make sure about getting the feedback not at the end of development but during the development process.

Answer 3

Defects are inevitable part of software testing. So we have to find a way to handle them along with regular sprint work. In our qa services company we follow below process to handle defect found during acceptance tests:

• While sprint planning we keep a certain percentage of developer's time(10%) in fixing defects
• We can triage all the reported defects and only include the high priority defects in current sprint and add the remaining defects to backlog
• While backlog grooming we can add the defects to upcoming sprints based on product roadmap and release priority

This helps us to cater sprint work as well provide fix to critical defects reported by testing.