37

What is your team's approach with those bugs that you open and report with the absolute certainty that there will never be time in the future to fix them?

Do you think it is better to keep them reported and in the backlog, for whenever (if ever) it makes sense to tackle them, but at least getting the visibility of "hey, we know this problem exists",

or

Do you prefer the approach of keeping the backlog clean, so if there is a bug we know exists but won't be dealing with in any near future, just mark it as rejected and move on?

I personally vow for option 1, but I am confronting people who lean towards 2 - which kind of discourages people to even open bugs in the first place - what for, if they will be discarded?

15
  • 7
    I’m voting to close this question because it belongs to project management and not to testing – Rsf Jun 15 at 6:50
  • 4
    I am not sure how "knowing what to do with low priority bugs" is not "communicating the status of a product effectively", imho. Take it from the position of "do we report minor bugs, then?". Also, there are many QAs out there taking these decisions as well. – ggonmar Jun 15 at 8:53
  • 1
  • 7
    @rsf it might not deal with testing but it would be silly to say it doesn't deal with quality assurance. – corsiKa Jun 15 at 14:48
  • 5
    I like to use low-priority "wishlist items/bugs" as onboarding exercises for new people. – Andrew Jun 15 at 19:21

10 Answers 10

39

Some opinionated points from my experience, doing mostly development and operations with only a bit of QA and support, for the past few decades. Make of them as you will.

I don't think it matters if bugs are picked up during formal QA, or by developers doing other work, or even customers or consultants. They should be treated, mostly, the same. My opinion is that they should all be recorded if they are serious enough that someone is motivated to point the bug out or report it.

  • Low priority bugs may later increase in priority (for business or technical reasons (e.g. other changes make the severity of the bug worse)).

  • Bugs can be incorrectly considered low priority and this can be noticed when e.g. support have a record that n people have called about a bug, or they have spent n hours due to the bug. If there isn't a ticket for support to find then they have to, in effect, create their own record to notice trends in calls and this work could have been saved. Support also don't always have the right perspective to accurately describe an issue in a way such that it will be found again when needed as they may only have a customers description - though that applies to everyone some of the time of course.

  • Working out what is wrong can take a lot of effort - more effort than a search for the symptoms (esp when you have a unique log message). Sometimes you observe something "odd" about a production system and you are unsure if its a symptom of something horrific. A quick search in the issue database to find that the problem is known and not serious can save a lot of time, especially at 4am after many stressful hours of recovering a system that encountered some mishap. I won't speak of the emotions of spending many hours of the night looking at an issue to find the next morning it was known but hadn't been logged and you could have had more sleep or spent time with your family.

  • It might not be obvious to everyone that an issue is low priority. People on-call may not be experts in every part of the system and may have just been pulled from their beds. They may well spend time diagnosing an issue and logging it only to have it closed. This is dispiriting for the reporter (speaking personally), especially in a product with a lot of issues. If there was already a record of the issue, while it may be disappointing that it wont be fixed, at least you haven't wasted time trying to write a good bug report and they might learn something (i.e. why it actually isn't a higher priority may give them insight into the system).

  • Sometimes developers don't have time/motivation for something big. Maybe they start holiday tomorrow and they just finished your planned work earlier than expected. They have a few hours now but don't want to start anything they can't finish or taxes their mind too much...they could just read the news till they think its safe to sneak out (so I've heard...) or maybe they could pick an easy but interesting bug and get some satisfaction of crossing something off. This requires an issue to be an easy fix as well as low priority of course.

  • A minor point but I have often used low priority bugs to train new developers on how the team works. There is no pressure on them to get the bug fixed by a particular time, or indeed ever, so they can concentrate on learning the SCM system, the build system, the ticketing system, whatever other archaic system inflicted on the dev team. If they manage to fix the issue it's a bonus.

  • Having lots of tickets shouldn't be a burden. If it is you need a better ticketing system or better method of recording of issues. Whether true or not, I have often had the impression that people who constantly worry about the bug numbers are never the people who actually fix the bugs, or the people arguing for more resources to fix the bugs, but are often the people who have to put the numbers in a power-point and pray no one asks them what they mean.

So my summary is record everything that has been found and described. It may help someone, someday, and the disk to store them is cheap. If you have people who are desperate for no open issues, suggest they fix a few bugs ;-)

1
  • 2
    This definitely echoes my thoughts - everything should be recorded if someone has noticed it's a problem. That's the whole purpose of keeping a track of issues. Wilfully ignoring them because you think a growing backlog might impact the psychology of engineers or PMs just says you're not educating people about what the backlog actually means, or you're not managing priority / severity properly. Low priority or relatively trivial issues should be backlogged and categorised: who knows, you may be able to use them for education (e.g. interns, new engineers) – AndyW Jun 18 at 8:16
19

There is a third way, a middle of a road way, if you wish: don't polute the backlog with many low priority bugs, but group them in an epic or a story that might hold them.

So, instead of having 20 low priority front end (for example) hot fixes that take 10 minutes coding each but you don't want to build a vesion for each of them, you can have a dedicated story that will take a half day of coding and another half day of testing. Of course, that is supposing those fixes are low risk as well as low priority.

8
  • 16
    I think you are confusing low priority with low effort. Just because an issue isn't that important doesn't mean it takes 2 weeks to fix. Actually that might be exactly the reason it will never get fixed. And putting many of those in an epic doesn't improve anything. – Jens Schauder Jun 15 at 8:15
  • 2
    I agree that low priority and low effort are not the same, but here is an option when they are. Putting them in an epic was a way of addressing polluting-the-backlog problem, just that. – Mate Mrše Jun 15 at 8:36
  • How well does this scale to production software? In companies that have software with decades of features built up over time, we see 100-200 bugs reported internally for our team alone per quarter. After mercilessly killing off every ticket we wouldn't do, not even 2 years later we had to close another 1000 – Drew Jun 16 at 4:00
  • OP says there's a certainty they will never be fixed, so why log them in the first place? – FDM Jun 17 at 5:27
  • 1
    @FDM I have seen "we will never fix this" bugs pop up years later impacting something that turned out to be important after all, and it was convenient to have a ticket around saying "we told you about this and you told us not to fix it at that time". – user3067860 Jun 21 at 14:59
13

You deal with them the same way as any other bug report. Review the bug and decide what (if anything) to do about it.

If you decide to do nothing, tag it in the bug database with "won't fix" and add a description of a work-round, if available.

That doesn't "pollute" your backlog. When you have decided not to fix them they are no longer in the backlog. But it's worth keeping a record of the decision, because a new user may rediscover the bug and report it again, and one day you might want to change your mind about not fixing it.

2
  • 2
    +1 for "when you have decided not to fix them they are no longer in the backlog". The answers promoting rejection or non-reporting seem not to have considered this point. – JBentley Jun 16 at 10:01
  • 5
    Don't forget to record the reason why you won't fix the bug. – Abigail Jun 16 at 13:10
13

I go with reject and move-on.

The downside is that other folks and new folks will keep discovering the bug 'anew' and have to remember them in their head. Which sounds like a huge problem.
In practice I have found that the fear is worse than the reality. Also consider that in many systems the number of bugs based on the combination of different devices, sizes, bandwidths, etc is essentially infinite anyway.
The other key factor is that just because the bug has been previously found and recorded it will still be found by new users, testers, etc. The difference now is that instead of logging a ticket, they allow the existing ticket to stand. However they still had to 'learn' what existing bugs are by actually encountering them. Reading through all the tickets in the backlog can help here but is rarely done as there may be hundreds or thousands and reading them all is not usually considered a standard practice for on-boarding new employees in most organizations that I have encountered. Not to say that you can't innovate anyway. Maybe a week of reading bug tickets will be something you decide to actually do.

The upside is a significant one - you move away from the dreaded 'ever growing backlog that everyone ignores' problem. The reason why this upside is worth the downsides to me is that after a while, tickets become a 'dumping ground' to park issues and not fix them and after a while this happens with more and more significant issues that are no longer as trivial as the ones that led to this practice in the first place.

I have seen this at multiple employers in a row. When I was finally able to manage tickets and bugs and a backlog myself I was determined to break this mold. I was successful and the backlog slowly diminished over time. However this meant a radically different approach to application development. Call it 'true agile' with developers actually truly empowered to make coding and design and maintenance decisions themselves.

I say radical because it requires processes that will be hard to introduce and sustain at many companies such as:

  • refactoring can often be 50-90% of the work for a sprint. Not "as well as as features" but instead of.
  • a focus on communication through multiple methods and on approaches such as 3 amingo to ensure that the team takes a dynamic and flexible approach to development, always willing to question assumptions.
  • issues have to generally be fixed now. Not "at some point in the future" (frequently never) but actually fixed now to the satisfaction of the product owner. New features and releases have to be truly balanced against bugs, refactorings, quality issues, test automation etc.

The fact that you can't record bugs and issues to get the satisfaction and benefits from filing them away may, conversely, in some organizations, increase pressure to fix things now. Also, if instead of 'filing a bug' someone has to talk to the developer and have a detailed conversation this will often be a better approach (and generate more ideas, solutions, workaround, etc). Not to mention empowerment. However this will depend on a ton of external factors about the culture of the company. Unfortunately in many and perhaps most orgs today, this might well lead to even lower quality if folks aren't already motivated to produce high quality. In those situations, not recording bugs will be seen as not caring about them either.

So this is complex, non-trivial and requires long-term vision and approach. That's where good leadership comes in. Hire them.

Remember that in organizations that claim to be "Agile" (like all of them today) you're supposed to fix things now instead of creating processes to 'manage them'. like ticket backlog.

Warning signs that quality may not be job #1 ?

  • a large backlog
  • a growing backlog
  • refactoring sprints
  • technical debt sprints
  • tech debt fridays

In other words, I would look at the big picture and think about:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

Sound familiar? :)

6
  • 2
    I understand - and to a point agree your "agile" approach :), but I could counter with the alienation that agile suffers in real world, with the straight-forward thought of "do we pollute the backlog, or close our eyes on the problems we know we have?" and from that perspective, it still feels like taking the ostrich hide-head-on-ground is the bad one... – ggonmar Jun 15 at 8:45
  • 3
    Yea, the purpose of the backlog is to keep track of bugs, not to be a sales brochure. There is no benefit to having it "clean". Perhaps just marking super-low-priority bugs as such is enough. You can also filter them out when showing the boss / some other marketing / spin-doctor presentation. – Reversed Engineer Jun 15 at 9:38
  • I have experienced significant benefits from a clean backlog Ymmv – Michael Durrant Jun 15 at 12:48
  • 1
    Psychology also counts. Marking bugs as super, low-priority, probably never fixed may, over time, lead to demotivated employees that don't care as much. That has played out several times in my experience. A better approach to closing ones eyes may be a cost-benefit analysis that will show, even if the issues can be fixed, whether it is worth it to the business to fix it. – Michael Durrant Jun 15 at 16:37
  • What is this "3 amingo" method you speak of? – Lorem Ipsum Jun 15 at 17:06
7

Personally, I ask the other team members if such and such a problem is something we even want to deal with. If we agree it's not, I don't bother opening a new bug because obviously no one cares and I don't like doing work that's considered useless.

I think what's important is that the team, the customer, or whoever decides what quality is with a concrete product, knows about problems. Over time, we (the dev and support team) learn that some problems are just irrelevant to the client or customer. Then such problems are not something we spend time on.

Personally, I like to save mine and other people's time, so sometimes I'd prefer to talk to developers in person, perhaps going over some issues on my or their laptop, rather than logging a new bug into a tracking system, which usually introduces a huge burden on me and my colleagues as well. But it depends on the context, how much time we have, how much time they have, how much we know each other, what the company processes are etc. There's no universal rule for that, but it's true that personal communication, rather than formal logging of bugs, can save a huge amount of time.

So, some quick points that might guide you:

  • learn what quality means in your context
  • and focus on that, ignoring the rest that is obviously not as valuable
  • that usually leads to not opening many minor "bugs"
7

I suggested this in the comments but figure it works just as well as an answer:

Keep the low priority bugs around, especially things that aren't hard to fix in principle. Use them as onboarding exercises for new people. They'll be just as good for learning codebases and team processes as anything else; delays from lack of experience won't disrupt anything important; and the new person will still get to contribute something genuinely useful.

2
  • But what if there are no new people coming that often? I like this idea, but sometimes seeing these low priority bugs issues in the backlog or issue tracker is a bit annoying. It makes me feeling like there are so many To-dos that are awaiting – konekoya Jul 2 at 1:02
  • You could filter them out in whatever view bothers you, I suppose. – Andrew Jul 7 at 20:31
3

Distinguish "low-priority-and easy" from "low-priority-and-hard".

For low-priority-and-easy (for example a spelling mistake in an error message) just fix it. It should be less effort to fix it than to manage a bug entry for it. Make sure your processes allow such bugs to be fixed with 5 minutes' effort; they don't even need a test case. Sometimes something will fail as a result (usually a misguided test case that's comparing the text of the error message with the existing incorrect message): pick that up if the regression tests fail, but don't create new tests.

For low-priority-and-hard distinguish hard-to-fix from hard-to-test. If the bug is on a path that you know is untested (for example some unusual response from an HTTP server), but creating a test is difficult, it may be worth replacing untested incorrect code with untested correct code. Or it may be possible to do a one-off ad-hoc test rather than a repeatable test that you can add to your permanent test suite.

For bugs that are genuinely hard to fix, assess the likelihood of occurrence and the impact of failure. If it's very unlikely to occur, but catastrophic when it happens, look at strategies for detecting the circumstances and failing more cleanly when it happens. But if the impact of failure is low, consider just documenting a restriction and moving on.

2
  • Its not testers job to segregate bugs based on dev effort. – Vishal Aggarwal Jul 28 at 23:27
  • Developers should be testers. As soon as you allow developers to disconnect from the testing process, minor bugs stop being fixed (because it's too much effort) and you end up with a product that is increasingly unusable. – Michael Kay Jul 29 at 9:09
1

There should be dedicated time for low priority bugs. For example, once a month team should review bugs and work one, or two days on all of them, including few low priority bugs.

1

Report the bug(s) and move on with your day, you did your job.

Devs should want to fix them now, because fixing them later is more work for them.

If the team collectively decides not to fix it, that is fine.
If a PM decides it is not a priority to fix, that is on them.
If the bug is found by a customer or snowballs into a bigger bug...
again, you did your job and the bug has already been identified and documented.
0

There is an third critical perspective which matters as per my experience as QA over an decade.

Lets step back,Why do we even have so many low priority bugs in the first place that we start bothering about them?

There might be an far bigger problem lurking in the background..

"There's no sense in being precise when you don't even know what you're talking about." -John von neumann

Sometimes as testers we get caught up in the "number game" of bugs.We forget that quality is more important than quantity.This happens for various kind of reasons, sometimes it's peer pressure or management pressure or political reasons or due to organization culture issues where added value is measured based on all kind of numbers & metrics.

We can't see the forest for the trees.

Lets step back one more step,why do we even test ? Testing is an risk driven activity fundamentally. We test to minimize business risk. A creative tester can find myriad of ways to bring a system down. However the real question is , "What is the possibility of that a real user would ever do that on production floor??" If it's highly unlikely AND we are coming up with lot of different scenarios like that then are we thinking effectively as a tester?

I think in those cases this may signify a bigger problem to be aware of: as testers are we able to see application from pure business domain perspective & its priorities as well and not just as an QA??

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.