How do I deal with poorly documented bugs?

Question

I sometimes come across bugs that do not have enough information to reproduce the bug, or worse, understand what the issue is in the first place. Usually, business analysts, rookie QAs & sometimes developers write nonsensical bugs like this. Contrived (for simplicity) examples -

Ex.1 I see a server error when I open this webpage, for this user. The username and password is xyz.

Me: Does the page give more info about the server error ? Did you check the network inspector of your browser ?

Ex.2 I open the ecommerce webpage & add items item1 & item2 to shopping cart. Then, I see that the underlying CartService's api call getCartTotal response has Item 1 discount=0%. This is wrong. In ItemPriceService call getItemPrice(theItem) we have Item 1 = {$50, discount 10%} & Item 2 = {$100, discount = 0%}.

Me : So, you meant to say that in the shopping cart, any discounted item is priced at its regular full price. (...and the rest of the info in your bug report is the underlying details of the bug)

Due to such bugs, I often have to waste time in chatting with the bug reporters to understand & reproduce such bugs. How do I deal with such poorly documented bugs ?

Some solutions I think might help. Do you see any issues in these solutions ?

• Save 2-3 poorly documented bugs somewhere & make current employees & new candidates try to figure out what the basic issue is & how to reproduce it. Then, show them the well documented version of the bugs to understand how much pain others go through in understanding the bugs. I will have to ask management to do implement this solution.

• Create general guidelines for bug reporting, perhaps even specific guidelines for api testing, UI, database etc.

• Reject the bug saying that its not clear what the issue is.

Answer 1

This is company specific. Any sort of bug reporting can work in different teams.

Your two examples seem kind of silly though. Whats so unclear about opening and navigating to web page, and reproducing error, or adding items to cart and checking what discounts are applied?

My personal opinion is that when dealing with new or inexperienced developers, one needs detailed bug descriptions, 'press x, then y, then z, got this, expected that' while when dealing with more senior ones who take bugs seriously 'shit doesn't work yo' is sufficient. Some even LIKE to chat about bugs.

If you don't understand how to reproduce the problem at all, just reject, reject. Reporter will get a hint after a few times.

Quite often testers don't have time too, to investigate every bug in detail, so they notice problem, quickly report and are off to find more problems.

If its a TEAM problem, bring it up in any of regular meetings, see if anyone agrees with you, bring examples and brainstorm a solution. Maybe you need more detailed steps, or more information than 'press X there is an error', and that's fine.

If it's a YOU problem, then reject what you don't understand, bring it up, but be prepared to adjust that there will be no resolution.

Answer 2

I'd establish good practices for quality bug reporting and I'd have a meeting / session / lunch & learn to present them.

I'd use this opportunity to create, establish and document how a bug should be reported.
I usually make sure to mention that they should consider capturing:

• URL
• Date & Time bug logged. Often automatically captured by the bug tracking system

• Steps to Reproduce This is the key section. The section name here can make a really big statement here when it is specifically called Steps to Reproduce, rather than just "description" or "details".
  In detailing Steps to Reproduce, I'd emphasis that These steps should guarantee the application developer can see the problem. If the problem can't be reproduced it will usually be harder to fix. I'm a former developer so this argument means a lot to me. It's also an easy sell being fairly obvious once mentioned. Although it's not impossible to find a bug if the error can't be reproduced at will, in such cases the company will need to consider gathering other information such as server logs, data transactions, etc. to try and troubleshoot a problem and it will often be much harder to diagnose the issue.

  • Here I would consider some positive examples in the presentation, i.e.
    1. Go to this URL
    2. Enter this data
    3. Click this button
    4. See incorrect data

• Device / Browser / Version

• Screenshot. A picture can say a thousand words
• Video. If a picture can say a thousand words, a video can say a million
• Data used
• Severity
• Frequency

I'd let any existing bugs that don't have all this to slide by. However when a developer starts working on one of the pre-existing bugs, if they can't reproduce it, they should now feel more comfortable going back to the tester and asking what are the steps to reproduce this ?

I'd also let simple typos and the like just capture the steps to reproduce and maybe a screenshot.

For your question about the three approaches, I would advise:

• Not pursuing the poor examples. Just focus on good practices and why they exist.
• Creating and publicizing the guidelines as I've documented with the reasons behind them
• Not rejecting a bug but making sure that a conversation can take place to clarify it as needed.

If patterns continue consider a group testing retrospective that you organize.

Answer 3

Poorly documented bugs should always be countered by defining a mandatory template for logging bugs.

Summary: Define the bug in 140-200 characters. Usually it should be in the following format; When you can't perform an action:

Unable to perform [some action] on [some module/page/component]

When there is incorrectness or mismatch:

Mismatch in [message/filename] when [trigger action]

Steps to reproduce: I recommend using Gherkin language to define steps.

Format:

Given a precondition
When actor performs some action
Then expected results

Example:

Given User has forgotten password (PASS)
When User clicks on 'Forgot password' link on login page (PASS)
Then Application is redirected to 'Reset password' page (FAIL)

Expected behavior: Usually the expected results as precise as possible.

For example;

The correct warning message to be displayed is #E4054: "Please enter valid credentials."

Actual behavior: Usually the actual results; as precise as possible.

For example;

The warning message displayed is: "EXCEPTION WHILE LOGGING IN! java.lang.AssertionError"

Build/Changelist Number: The build number or changelist or version of the software being tested in which the bug was found.

Environment The environment in which the bug was found. Can be Test/Development/Staging/Production, etc.

Attachments: Screenshot / video of the steps to reproduce. Logs that are generated in case there are no screenshots. Any other files to support the existence of bug.

Answer 4

If you are finding bug reports from QA engineers to be lacking enough clarity to reproduce and fix, this seems like an issue to bring up to the QA Manager. Are new QA engineers not being properly trained? Are more senior QA engineers not properly overseeing the work of newer engineers long enough?

Your first two bullet points seem like good ideas, but I'd avoid going directly from QA engineer to QA engineer. It should be a team effort between the Dev and QA management teams.

If you truly can't understand a bug report, rejecting (or if your system allows it to be put in a feedback state that throws it back to QA for review without actually rejecting it) it is fine. But doing this is unlikely to actually address the larger issue and make cause unnecessary friction between teams.

Answer 5

Team effort should do the trick.

For each project you should have a test strategy, you should also have a guideline with how a bug is reported and the flow that a bug has.

Reporting a bug is an important activity, a poorly written one can result in a lot of time spent digging in the wrong places.

Each bug need to contain details enough to reproduce it and to have a clear description.

What thing you can do to improve this:

• talk with the person that reported the bug and ask nicely for more details
• if no guidelines for adding a bug is documented and you think is needed then ask nicely in a meeting for this
• if this happens often then you should talk with the QA lead from the project, else the Project Manager, this should be handled by the team
• make use of meetings like scrum and retrospectives and others to see what anyone can do better to increase the value and the efficiency of the team
• try to solve this issue in such a way to make a better team
• make small gesture if the bug is updated with better info, like say thank you once in a meeting to that person for the effort of writing better description for bugs, leave a sticky thank you note on the laptop, sometimes small gesture can do much

Things you should NOT do:

• talk directly with a person that has 2-3 level above, like QA manager, remember you are a professional and a team player and should solve this inside the team
• simply reject the bug if you do not understand it without trying to improve the communication in the team
• showing the bug in a meeting and ask tell how to reproduce it or explain it

Answer 6

I like your solution to show and discuss the best. Let people experience what you experience. And ask them to help you so you all can do a better job. Be aware of the fact that not everybody understands your problem as it might seem 'logical' what is wrong. The best way to illustrate that, I think, would be to show a vague/ambigious report and ask people to describe, for themselves, what they think is wrong. Then let people share their ideas. The more they differ, the stronger your example and case might be. Use that as well for creating your 'teaching material'.

Guidelines might help if they support the thinking about what and ho to report. If people use them as checklist it will become totally useless.

And rejecting is always an option. Try to be provide information that let people help to understand why it is unclear. Otherwise you will become the guy that is always 'negative'.

As a side note: I think chatting with the bug reporters is part of the job. Having close communication is a great way of getting more & better information.