Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Currently automated reports run on recurring monitors based on user encounters. At the start the automated errors were for the most part useless because there were 400 low priority bugs, 50 major, two important, and one critical. While users would show 20 low priority, 40 major, 15 important, and 20 critical errors. More specifically, errors reported from users and monitors would produce the same report. Example: User3's automated report encounters first then maybe 5 minutes later user3 will show the same report.

What are some of the best practices/methods to normalize these reports and delimit redundancy? I've considered running a QA team through it then use one run auditing based on
-Category
-Description
-Date
-Error Code
-Included Image
-Objects
-Tags

share|improve this question
add comment

2 Answers 2

I'm not aware of any standard methods for determining which bug reports are duplicates. That said, the method I use works more or less this way:

  • Group and categorize - I start by looking at where in the application the problem occurs and what kind of problem it is. It helps here to have some knowledge of the source code: if you know that problem X occurs with a shared component, you can reasonably group reported problems with similar reproduction and results as duplicates (for instance, a shared list view that errors out on sorting by ID no matter where the list is viewed may report as multiple issues via an automated tool but a human who knows the list view is shared will treat it as a single issue).
  • Flag duplicates - How you do this will depend on your issue tracking and reporting system. Most of them have some method of flagging a duplicate. Most places I've worked the norm is to close the duplicates after flagging them, but that's a procedural thing. The important part is to link all of the duplicates in some fashion so that it's clear they are duplicates.
  • Flag child/related issues - often a problem will be related to another problem in some way. For instance, a bug that allows a user to save records with invalid data will cause problems where another part of the system is using the data. An automated reporting tool will generally report these as separate issues, but they're not, strictly speaking, duplicates. As an example, if an invalid date of birth can be saved, and the module that sends a birthday reminder doesn't have handling for invalid birth dates, there'll be errors for the record with the invalid date of birth. These two issues should be related in the issue tracking system, but they're not duplicates.
  • Consolidate similar issues - This is more of a convenience than a duplicate reduction tactic. As a general rule, minor GUI/interaction problems that an automated system will report as separate issues can be consolidated for convenience when they're all occurring on the same form/component/page. For instance, I'll typically list all the spelling errors and layout problems on a single screen/form/page as a single issue because they'll all be fixed and tested in a single pass and it saves on administrative time.

There'll always be a judgment factor, and you'll never eliminate all the duplicates, but this is a way to help limit redundancy and duplicate issues.

share|improve this answer
add comment

What you appear to be asking is if there is an automated way to reduce redundant defect reports before a human reviews them. First, look at the opportunity costs associated with that approach:

  • How much time does it take a person to review each report and perform an initial triage? For a person familiar with the existing defects, product function, and code structure (ie, an experienced tester), that should take much less than a minute for each report. That estimate includes a quick search of your tracking system for key terms to identify duplicates.

  • For those reported issues that are more involved, intermittent, or unclear to a human, I doubt that an algorithm can be created that will accurately identify those issues faster.

  • Now consider the impact of overlooking a serious, unidentified issue that (on the surface) appears to be a duplicate and is subsequently mis-categorized in your tracking system. Even if it is left open, it will probably be closed out of hand as a duplicate when the primary issue is closed. As a result, that initial classification is critical.

If your defect backlog is so massive or your defect reports are structured such that duplicate searches take several minutes and are inconclusive, then I would suggest working on improving that instead.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.