How to tackle a huge bug backlog?

Question

What is the best approach in dealing with a huge bug backlog? Currently we have a huge bug backlog and i was looking for an efficient way to deal/categorize/ how to manage the backlog?

Answer 1

This is very common.

There are basically 3 parts to the problem:

1. Measure.
   Track stats to know when the backlog is getting worse or improving, week to week
2. Identify.
   Figure out what things you need to change to stop making it worse week to week
3. Cleanup.
   Clean up the backlog you created, bit by bit

I'll focus on part 3 - the cleanup - but if you don't do parts 1 and 2 you'll quickly be in a bad spot again (if you ever get out at all). If you want more info on Measure and Track then ask questions about those parts specifically for specific answers that can go into detail (and solicit a variety of answers of course). Also Kate's detailed answer has good info for those activities.

More specifically, for the cleanup, I would look to

• Prioritize. They can't all the the same priority. Maybe rank into High, Medium, Low.
• Estimate difficulty. In terms of relative complexity to other tickets. Using points and voting is one approach.
• Categorize, Tag and Label. Some tickets are minor, say layout issues, others might be significant, maybe database issues. Creating categories and using tags and labels can help in triage.
• Use an Agile approach focused on this weeks (or this sprints) work.
• Use Big monitors to show the backlog constantly.
• Use a tool like Jira, Trello or Pivotal Tracker to manage the tickets.
• Mix current development tickets with ones from the backlog.
• Delete old tickets. Even "a good idea", "we should do" will often not be done. Be bold in your deletions. You can always recreate or revive tickets
• Write tests. If you have accumulated a lot of bugs, are folks writing tests to prevent them in the first place with TDD/BDD ? If not they should start now to prevent the bugs increasing.
• Use QA/QE to help test for, find, prioritize and categorize bugs
• Review the amount of work assigned to, done and overflowing from sprint to sprint and see if you are simply taking on too much work.
• Bring the issue up in retrospectives to get more ideas on how the team would like to address it.
• Think short term and what resources would it take to actually address it right now compared to those available and current feature development
• Think long term about how to avoid getting back in the same situation. Stats on tickets (added, resolved, outstanding, etc.) and the trends week to week should be monitored. Long-term thinking should include analysis of how things got the way they are right now.

Answer 2

In addition to Michael Durrant's excellent answer and the equally good comments, I'd suggest you consider a few things:

• If you have not already done so, devote some time to analysis of your bug backlog. You will probably find some combination of the following things:
  • The bugs cluster in certain areas of the application. These will typically be the areas that are most complex and/or had the least design and architectural attention.
  • A significant number of bugs are caused by insufficiently defined user stories or missed user stories.
  • A significant number of bugs are in areas that are not unit tested or unit-testable.
  • A significant number of bugs are regressions of previously tested functionality (which is probably not tested by automation or not testable by automation).
• Once you have a decent analysis of your bugs, you will find that they point to a number of areas where your process need to improve. For example:
  • If a lot of your bugs are customer-reported, your team probably needs to devote more time to integration and end-to-end automation, starting in the parts of the application where customer-reported bugs are thickest.
  • If a lot of your bugs are regressions, your team probably needs to devote more time to building good unit and integration tests and maintaining them as functionality changes.
  • If a lot of your bugs are obsolete (that is, have been fixed in later versions but were never closed) or duplicates, your team probably needs to spend more time grooming your bug backlog along with the feature and user story backlogs, as well as aggressively hunting duplicate reports.
• When you have an idea why your bug backlog keeps growing, where they're coming from, and the root causes of them, your team may need to designate a sprint as a bug cleanup sprint, where you start with the highest priority bugs and work your way through the bug backlog.
• For a really horrible bug backlog, you might want to designate every fourth sprint (or whatever cadence suits you) to bug cleanup until the backlog is under control.
• If insufficient unit/integration/end-to-end test automation is part of the problem, you may want to devote one or more sprints to fixing the test automation.
• If you are not able to devote sprints to automation or bug cleanup, then chances are the problem is higher up the food chain than your team and needs to be addressed with your management - a team that is not able to spend the time they need on quality practices or reducing their technical debt (which your bug backlog represents) is typically being pushed or required by their management to produce more new functionality than is sustainable (in which case I sympathize. I've been there, and educating management is not fun).

Answer 3

When I have worked on resolving bug reports in the past I found that the backlog was able to be shrunk due to a large number of people reporting the same bug. We wrote a similar bugs plugin for our app which searched all open bugs and recommended which ones to link as the same bug, then a new bug report was created automatically with details from all the customer submitted bug reports and the customer submitted bugs which where for the same bug where all linked together and related to the one bug, in one instance we where able to take nearly 1000 bug reports and reduce them to a single master bug report which had been duplicated so many times simply because the bug was in a frequently used feature which was bringing up an error display and our error display featured a "Report as Bug" button.

Answer 4

Big long lists of bugs are a waste of time. Bugs will become harder and harder to reproduce while you are going over the list over and over again. I once had a list of 3000+ defects, with no end in sight.

I will suggest something radical. Throw all the defects away and introduce a zero-bug policy.

Now when a new defect comes in:

• Triage it according to the policy. Fix critical stuff now, regular bugs after your current work. Decide if all others are features, improvements or trivial's. Only place them on a backlog if it is really worth it in the next period. Communicate clearly to the reporter.
• When fixed do a root-cause analysis and tackle places where bugs are created. Improve the process and the code-base.

The Agile team should try to keep fixing all bugs as soon as possible. It lowers their velocity, which now becomes honest, due to past mistakes new features will come out slower. Inspect and adapt your planning with this new knowledge.

Answer 5

Faults are an inevitable part of software delivery. I only briefing touched on managing lists of faults in Agile Project Execution so it is well worth expanding on.

In fact defects is an area where software delivery, including agile software development, can get messy. Luckily there are ways to manage this mess or to avoid it altogether. Jo’s question hints at potential answers but there are others. In fact I know of at least nine common strategies for managing a growing fault list.

My strategies are:

Do nothing
Filter out the noise
Stop logging low impact defects
Close off low impact defects
Prune the backlog
Batch the defects
Blitz the defects
Fix some every sprint
Automate the tests

1. Do Nothing

Before deciding which option to take you’ve got to know who cares. Who cares that the product backlog is large? Product Owner? Developers? Project Manager? If nobody cares then it isn’t a problem.

2. Filter out the noise

Then there is, why do they care? Maybe your developers care because they see a lot of noise in the product backlog, things that they shouldn’t be working on. If that is the problem then just make sure the developers don’t see the low impact items until those items get prioritised. If your product backlog is electronic then you can use some kind of filtering for this. If the product backlog is cards on a wall, then put the non-prioritised cards somewhere else.

It might be your product owner who hates wading through low value defects when they prioritise the backlog. Filtering can help here too.

Maybe the tester cares because it makes the fault statistics look bad. In that case check where these figures are being reported and see if any anybody else cares. If the higher ups don’t care then persuade your tester not to care. Alternatively negotiate reporting something else, e.g. only high and medium impact defects and exclude low defects, or report number of defects in each category. This is also essentially a filtering solution.

3. Stop logging low impact defects

If you have to do something to reduce the number of defects on the list then you can, as Jo suggests, just stop logging low impact bugs. The only trouble with ignoring low impact defects in this way is that throwing the card away doesn’t fix the problem. There is a risk that people will keep noticing the same faults and raising them again. None-the-less you might find that risk worthwhile.

4. Close off low impact defects

This is a bit similar to not logging low impact defects at all. In this case you don’t fix the problem but decide it is never going to be fixed and just close the ticket. Same risk that it’ll be raised again, but might not matter.

5. Prune the backlog

Even if you retain the full list of defects you’ll have to prune the product backlog occasionally as the list will become stale. Over time, because of changes to the system, some items will become irrelevant. Irrelevant defects are just clutter and should be purged.

6. Batch the defects

Another option is to batch related defects together. They might be low impact and low value individually but in combination their overall value will increase. For example you might group together a set of “email” defects and get the product owner to priortise these grouping rather than the individual items.

7. Blitz the defects

A common tactic is to blitz the defects. That means allocating the whole team to fixing defects for a sprint. This often happens immediately before a major launch.

8. Fix some every sprint

It is also common to allocate a certain proportion of effort to fixing faults each sprint. I advocate giving the technical team time each sprint to fix technical debt and fixing faults can be rolled into that.

9. Automate the tests

The last and best of the nine strategies is to push automated testing. The goal with automated testing is that all defects are addressed within the sprint thus the number of defects that need to be logged are vastly reduced. And the step of logging the defect can be often replaced by writing a new test.

Answer 6

If you don't have a big backlog then you don't need to deal with it. I think to manage what you have currently, in addition to all the good points already presented to you by others, you have to figure out the phase(s) most defects are injected too.

In my understanding of the Agile theory, defects are technical debt, you don't really allocate time in a sprint to work on technical debt. Each sprint has a clear goal -- working on stories. I know in reality teams may allocate resource on the side to tackle on technical debt, but this means it should at the same time take away resource from the sprint (lower velocity), and this is rightly so to avoid more defects injected due to lack of proper resource/time.

From my experience, unless your backlog is from legacy code (days before being Agile), you should look at your "done criteria" for your Agile stories. When a story is done, it should be "done done", not just code written ready for QA. If too many defects come our of these story, you should make sure necessary testing is in the storys' done criteria. Hopefully your backlog will shrink this way.

Answer 7

In "Perfect Software and other illusions about testing" Gerald Weinberg proposes declaring a "bug-amnesty:

Remove all bugs from the bug database and send a notice to all issuers telling them to resubmit a bug if they're still concerned about it. […] When I did this with the 14,000-bug database, the reduction was enormous and the problem was solved.