Should you close old bugs?

Question

If you find more bugs than you fix, then what should you do with the ever expanding bug queue?

The queue can grow for several reasons:

• The bug is deemed too low priority to be worth fixing
• The bug is edge case that has a low probability of happening
• The business is prioritizing new functionality over fixing issues and is willing to accept this form of debt
• You're either not finding or fixing bugs during development

It has been proposed in my team, that we close bugs older than x days, as they will get re-submitted if they're still an issue. Assuming you prioritize the higher severity bugs and try and fix known bugs in new changes, then this doesn't seem like too wild of an idea.

Answer 1

I would say that product owner (or whoever ultimately decides what the team works on) can decide that some bugs will not be fixed for one reason or another. Then I would close those bugs documenting this fact so that later it is clear what has happened.

I wouldn't close them just based on time as I see it should be conscious decision to close them and time itself isn't good enough indicator.

Answer 2

I would like to consider each of the cases separately:

1. Low Priority - Low priority bugs do not stay low priority always. They always have a tendency to become high priority if enough of the users complain (or a important customer raises it). Closing doesn't seem to be a good option.

2. Edge Cases - Remember Murphy's law. These tend to happen and they happen when you least expect it. Closing them is not an option.

3. Business Priority - Even when the business priorities are different, bugs still exist. If the new version of software is green field, these bugs might not make sense - but still need to be verified.

4. Not finding or Fixing bugs - Time to talk to the development team :)

As you can see, I am not for closing the bugs till they are fixed. What I suggest will be to use the priority and category fields in tracking tool and keep them hidden if needed. A bug entry takes time and effort - and it is waste of energy to close them without resolving them.

Answer 3

Bug is Bug , it does not matter how old it is. As a tester you should not close bug if it is not resolved yet by developer and verified by tester.

Close old bug without solve and insert it again that will just increase bug counts. It is fine that you concentrate on high priority bugs first but to close without resolve seems wrong.

But if that bug is from requirements which is removed or totally changed then tester can take approval from client in written that because of this requirement change , particular bug is meaningless and we should close. In this way you can close.

Answer 4

In my experience, closing a bug simply because it's old is not a good idea. My organization used to do this as a policy, but it caused a number of problems.

• The bug can look like it's fixed. Test teams would run queries to see which bugs were closed between two releases. If a bug was closed, that test case was no longer run (since the issue was presumed to be fixed) until the next major release cycle. This resulted in old, typically low-priority bugs frequently disappearing and re-appearing. To a manager, this can even look like your team keeps making the same mistake over and over again.
• You lose potentially-valuable debugging information. A new bug will be filed if the issue is encountered again, but the new bug report is not connected to the original bug report. For bugs that are poorly understood or that only reproduce infrequently, you need all the information that you can get in order to debug it. It's usually more productive to add more information to an existing bug report than it is to file a second bug report for the same issue. For most bug tracking databases of a non-trivial size, it's not practical for a submitter to manually dig through all of the closed bugs for a possible match (it's hard enough getting them to look through the open bugs).
• It can encourage people to ignore problems. Have a problem that's not mission-critical and that has a particularly difficult or unpopular solution? Just ignore it long enough and it will go away on its own. This is definitely not what you want to encourage your developers to do.
• Hiding problems is almost always a bad thing. The only thing worse than having a hundred old bugs hanging around is to have a hundred bugs that you are no longer aware of. This will almost guarantee that they'll never get fixed, and won't prevent you from making the same mistakes again.

After a while, my group re-evaluated our policy and determined that it was making things worse, not better. We changed our policy so that instead of closing out old bugs, we had periodic meetings (after the end of each major project release/milestone) where we re-evaluated bugs that were over a certain age. This helped us understand why these old bugs existed in the first place, and why they've been able to stick around for so long. Many of them ended up being closed out, but for specific, meaningful reasons (e.g. the bug report didn't have enough information to start investigation, the test case was incorrect, or because the associated feature was cancelled and the issue is now moot) instead of simply because they were old. After a few rounds of this, our organization got better at identifying these sorts of bugs and changing our processes so that they didn't get filed in the first place. We were able to actually solve the problem of a large bug backlog instead of merely hiding it.

All this dances around the big issue, though. If you're creating bugs faster than you're fixing them, then you're headed for major problems. You really need to re-evaluate your development practices and make sure that you're improving the quality of your code base over time, not making it buggier. If you have that many bugs that are low-enough priority to ignore for an extended period of time, then it could also be a hint that your test plans are focusing on the wrong things.

Answer 5

I would not close bugs - you can "park" them - but it shouldn't look like the bugs are resolved. In our software release cycle we have ~2 months bug fixing phase - in this phase most developers are only fixing bugs no new features. This lets us get rid of most low priority bugs. And makes the product more "round".

In the end this helps you to ship a better product.

Answer 6

I think coming up with a good strategy of closing defects is very wise.

I once worked in a team which had over 3000+ major, minor and trivial bugs of which most were at-least 2-3 years old in the tracking system. At one point we wanted to start cleaning this and started fixing them. The result was hours and hours wasted on badly written defect reports that we could not reproduce either because it was already fixed and/or just not correctly described.

Just as never ending lists of features can be demotivating so will long lists of defects. You do not want to be alerted again and again for items you are not going to fix in the near future. Figure out what needs focus for the upcoming 2-3 months and plan accordingly.

Keep the blocking and critical defects at all times, but decide if you are going to fix the Major-Trivial defects and if not within a release-cycle time period archive them.

Personally I would try to get teams to go with a "Zero-defect" policy, do not create technical-debt and really understand how much new features (without defects) the teams can deliver.

Answer 7

When you close a bug, there should be a reason for closing the bug. The most obvious, of course, is fixed, but could not reproduce, not a bug and won't fix are equally valid.

Could Not Reproduce means exactly that - the person assigned to fix the bug has followed the step to reproduce the bug (your bug report does include those, right?) but cannot get the bug to occur on their development system. This status doesn't mean that the bug doesn't exist, merely that there are additional steps or conditions for the bug to occur; I have, for example, had bugs that occurred only under very specific temperature conditions. If this bug is important - see below - then it needs to go back to the testers to isolate the true conditions.

Not a Bug means that the behavior is not a bug, but that it is working as designed. You'll often see this when testing games - the tester believes something different should happen, but that is not what is in the design document. The tester might well be right, but testing is generally the wrong time for major design changes.

Will Not Fix is similar, but rather than working as design, it's a minor problem that won't greatly affect most users.

Keep in mind that not all bugs are created equal. Those which crash on start-up on every platform have the highest priority, while those which cause an insignificant graphical glitch under unusual circumstances would get a much lower priority. Programmers need to concentrate on fixing bugs based on their priority; as time goes by and the dreaded ship-date approaches, entire categories of bugs will be placed into the will not fix state. These bugs might be fixed in a later patch, but not right now.

So who assigns priority? Initially, the testers will, using some form of checklist or rubric. Next, the QA lead & programming lead should discuss new bugs before finalizing the categorization and assigning them to programmers to fix. The priority isn't fixed; additional testing might indicate the bugs are more or less serious, or occur more or less frequently. And, of course, as the delivery date approaches, some bugs seem far less important...

Once a bug has been resolved by the programmer, it still isn't closed. If it's been marked as fixed, then it has to go back to QA to verify that it has been fixed. Could not reproduce can either go back for more testing or left to sit if it's unimportant. Not a bug will typically need some confirmation that it's working as designed, while will not fix generally needs some sign off from upper management.

Answer 8

There are two issues in the question: how you use your bug tracking system, and how your deal with having too many bugs.

If you don't plan to fix a bug, you might as well make that apparent in your bug tracking system; these systems work best when they reflect reality. Any decent bug tracker will let you record whether the bug was closed because it was fixed or because you chose to ignore it.

There are lots of reasons why you might have more bugs than your developers can fix. Some reasons are actually software problems (e.g. developers writing too many bugs); some may not be (e.g. your testers are too picky, or lots of the bugs are duplicates, or lots of bugs are false positives.) Different organizations (at different times) have varying tolerances for bugs, too. If your product is too buggy, your users will let you know, and you'll realize something needs to change.

Answer 9

If you find more bugs than you fix, then what should you do with the ever expanding bug queue?

It has been proposed in my team, that we close bugs older than x days, as they will get re-submitted if they're still an issue.

It's a bad idea, depending on what "Close" means in your context.

If "close" means "we won't have to look at that bug any longer", then you have to assume the same issue will pop up next time you test significantly. Then, someone (the tester?) will have to decide "should I submit a new big report, reopen the old one, or just ignore the issue entirely". Each of these three alternatives has problems.

Seek a different status than "Closed". Perhaps "Deferred" or "Later" exist in your bug tracking system. Those statuses might mean "we know about the bug, aren't going to fix it in this release, but will re-triage for the next release". That would be my preference.

Then, spend some time on diagnosing why you have so many unfixed bugs, and work on a solution for that problem.

Answer 10

Despite being an edge case, or low priority, if you can still reproduce the reported bug, and the expected functionality of that system still dictates that it is a bug (instead of just an old requirement which is no longer valid), then it's still a bug, and should be left as so.

I wouldn't say its the QA's responsibility to maintain the product backlog (the product owner would do this) but in my opinion, a valid bug is still always a valid bug until it is fixed, and leaving these items in the backlog can provide good insight into the overall state of the software under test.