How to establish better tracking of escaped bugs?

Question

At my company, we don't have good tracking for escaped bugs (or awareness of in QA a lot of the time) once releases go to prod.

When the users find bugs they have a tendency to go straight to the developers and the developers don't tell the testers about it until they have a fix ready.
The problem with this is that there have been many times when it wasn't actually a bug, but a missed requirement or a requirement that was too ambiguous and wasn't coded and tested the way the user intended it to work. Both of those things need to be tracked too, but from a QA metrics perspective, they aren't being caused by something we missed in testing.

The other problem for me as a tester is that if I missed something I want to know what it is for any number of reasons.

Have other teams had the same experience? How did you remedy it? Our users don't have access to open bugs (we use TFS) so that isn't an option.

Answer 1

This is process related - nothing you can solve as regular QA tester.

Your QA manager need to talk to DEV manager how to improve communication between devs, QA and customers, and how to track relevant info to gradually improve your whole process.

Bug tracker is just one part of it. But as QA, you cannot make developer enter a bug to tracker, or do anything - only their own manager can do that, and only if s/he will see any value from that improved process.

Your QA manager needs to do that either with dev manager, or the higher boss above both of them.

And you need better argument that "you just want to know if you missed something". You (or better, QA manager) need to explain how establishing such process would benefit the whole company, how it improves the quality of your product and overall customer experience, not just solves something for QA.

One thing what should help you to get to your goal is that what you propose is considered industry standard best practice :-)

Regarding your customer's access to bug tracker: You should track customer feedback, but not in bug tracker. Don't offload to the customer tracking the bugs, dupes, blockers etc. Customer does not, and should not, care about your inner process. Use separate feedback tracker. In the bug related to a feedback (with better description than the user provided, result of your research) mention relevant feedback. Or many feedbacks, if reported multiple times.

Answer 2

Ideally when end users find a bug testers team is informed before developers. Then after it goes to developers because testers are to be blamed for leftover bugs :-). You should make some arrangement that users does not report directly to developers. They first consult testers for open bugs. Then testers obviously get informed before developers.

Answer 3

Someone needs to enter these bugs into the bug tracking system. If the users can't/don't, then someone must do it on their behalf.

Developers shouldn't be acting on issues until after a bug report is written.

If there is a Help Desk, then the Help Desk should be entering bug reports on behalf of the users. If there is no Help Desk, then Developers should be entering these bug reports. And if everything else fails, the issue should be passed on to QA to write the bug reports.

No doing any of this is just silly. Management should educate folks about proper reporting of bugs.

Answer 4

In quality assurance services, tracking of missed requirements and customer issues is one of the factors to ensure future process improvements within the team. Moreover, effective tracking of defects is only possible when a team has a defect reporting tool accessible to all the QA/Dev/Product teams, etc. within a project.

Often a software qa company follows a standard of analyzing the release and create a list of issues faced by the customer on the live env, also known as post-release defects.

In post-release defect analysis, the customer defects are tracked and shared with the other teams for further examination, and based upon the findings process can be improved for future release. It results in easy identification and tracking of a requirement miss or a QA miss.

To manage a requirement miss issue:

Either before or during the feature development, the QA team can start reviewing the feature requirements from an end-user perspective and update the dev/product team accordingly.

For a QA miss issue:

The QA team can collaboratively take measures to avoid its occurrence.

Post-release defect analysis is a team effort. It can be performed either by the Dev team or QA team, later followed by sharing the reports through emails or discussions during retrospection meetings with the team.

Answer 5

So, we encounter this issue with our customers but we have a platform where they report these live issues. It's then forwarded to QA to reproduce. Once they do, they log it into the test management tool we use Kualitee and assigns it to the dev team. It has an independent issue tracking module.

Then whatever communication takes place it's on the tool and auto email notifications to defect viewers.

Answer 6

Users should not be able to directly interact with developers. There should be a platform on which user can log their complaint. This can be a web based logger or a customer complaint agent. Someone from the testing team should convert that complaint from user language to a formal bug report.

After that, it can be treated like a normal bug: fix, not-a-bug etc.

The testing team should also review it to see if this should have been caught by the testing team. If yes, then they should incorporate it in their test case repository for future tests. TFS is very extensive but it can be rather inflexible at times. If you face such challenge, you can try using Kualitee. We have been using the tool and has been useful for such use cases.

Answer 7

The other answers seem to be only half-way good.

The biggest improvement is:

• install a bug-tracking system, agreed by the company;
• QA to start using it and add all bugs and problems as they appear.

Then it is the job weekly reporting to push the developers to act and fix the bugs. Otherwise, the management will see weekly an increasing pile of older and older bugs, with no bright future at the horizon.

If developers do not fix the bugs because QA does not report them, then it is entirely the fault of QA. So it is of top-most importance for the QA people to start using the bug tracking software.

With other words, the QA people are either the source of the problem, or the source of the solution - depending on whether they hide the problems, or report them.

---

Note: ideally, the bugs are found and reported before they "escape" into production. However, even after "escaping", the bugs have to be recorded and tracked. Actually, the escaped bugs should have higher priority, since they are already (potentially) visible to the customers.

Answer 8

Use a bug tracking tool and tag those bugs with a separate and specific test. Or you can create a separate suite for such bugs. I recommend using, Zephyr, Xray or TestGear for test case and bug management tool.