0

I have read several articles by Capers Jones, who states that:

Defect Detection Efficiency (DDE): percentage of defects found after release
Defect Removal Efficiency (DRE): percentage of fixed defects when compared to customer reported issues.

In the further text, he gives an example of DRE: when the development team found and fixed 90 defects, and the customer then finds 10, DRE=90%.

There is a note about the difference between DDE and DRE, stating that some companies choose not to fix all found defects before release, making DRE thus lower.

But how are those unfixed issues reflected? Let's say I found 100 defects and fix only 50. The customer, for some reason, found only 5. Following the stated process, DRE then should be 50/50+5=90%, even when I left 50 defects unfixed.

Is that truly correct? To me it seems counterintuitive that when I leave 50% of defects unfixed, in a situation when the customer found nothing, DRE would be 100%.

2 Answers 2

0

Bugs that are not fixed are comprised of many categories:

  • they were unknown
  • they were were not found
  • they are viewed as features
  • they are not well understood
  • they are not important enough to fix
  • they are not worth fixing based on the effort required
  • they are not worth fixing based on the life span on the product
  • they are not worth fixing because customers expect them and use workarounds

Using only the categories of "we found them" vs "customers found them" is too crude a measure for improving quality. Such a simplification will not address the complexities of the underlying issues and will also likely lead to human manipulation and misrepresentation to achieve other personal goals.

1
  • Thanks. You are correct, though I am just trying to understand the formula, which is IMHO quite vague (quote): "Defect removal efficiency (DRE) is calculated by keeping accurate counts of all defects fixed during development. After release all customer-reported bugs are included in the total. After 90 days of customer usage DRE is calculated. If developers fixed 900 bugs and customer reported 50 bugs in the first three months then DRE is 95%." QUESTION: If there were additional 200 that were not fixed, are those included in the denominator, too?
    – Alarias
    Mar 28 at 13:52
0

Stephen Kan's Metrics and Models in Software Quality Engineering Second Edition cites Capers Jones' Programming Productivity and states that Defect Removal Efficiency is not related to fixing the defects, but simply finding them. The equation presented is Removal Efficiency = (Defects found by removal operation) / (Defects present at removal operation) * 100% = (Defect found) / (Defect found + Defect not found or found later) * 100%.

Using a concrete example: If the development team finds 90 defects, fixes 50, presents 40 as known issues at release, and external stakeholders find 10, then Defect Removal Efficiency would be 90%.

It may appear that this makes Defect Removal Efficiency and Defect Detection Efficiency the same. However, I've usually seen the Defect Detection Efficiency with an additional part of the definition: it considers the defects that could and should have been found by an activity. If you consider the entire development process and have a robust enough process, then perhaps you should be able to find all of the defects. However, if you look at it activity-by-activity, some defects will be more likely to be found in different activities. For example, a code review or inspection may find an automated test that doesn't conform to the requirement which could hide a defect in the implementation, while dynamic analysis may find an injection vulnerability - you probably don't want to put finding injections on the code reviewer since that could be very hard, so if that defect escaped into production, you may find that your dynamic analysis is insufficient.

I would agree with Michael Durrant, though - just keeping track of who found a defect is insufficient. Especially when you consider iterative and incremental development models. There are plenty of good reasons why a defect could be found and not fixed, so getting into understanding things like why defects weren't found or why a found issue was not fixed is often more important and more interesting.

6
  • Thanks, yes, but Capers in his paper strictly distinct between DRE and DDE, where the former is about finding AND fixing, the latter just about finding. In your example, following Capers original definition (defects fixed), the DRE would be 50/60=83%.
    – Alarias
    Mar 28 at 13:43
  • @Alarias Like I said, DDE is not just about finding the defects - it's about finding them where they could or should be found. I wrote about this in the third paragraph. Not all activities are capable of finding all defects. Mar 28 at 13:53
  • Thanks, well I am really just trying understand the original formula, and how it works when there are defects that were not fixed. Capers mentions that often but all the examples he gives are trivial where DRE=DDE (90 found and fixed, 10 reported = 90% DDR). He defines DRE as "percentage of bugs found and eliminated before release". But it is unclear what he means - percentage of what? All defects found + customer reported? Those fixed + customer reporetd?
    – Alarias
    Mar 28 at 13:57
  • @Alarias It may help if you cite the exact source. Capers Jones wrote a lot of things. Unfortunately, I don't have any of his first-hand works about DDE/DRE, only things that cite his work and they tend to not agree with what you are quoting. It may also be interesting to note that a lot of Jones' work (at least that I'm most familiar with) is in critical systems that usually can't release with known issues/defects. So some of these concerns no longer matter, since finding the defect requires it to be fixed. Mar 28 at 14:25
  • One of them here, for example: ppi-int.com/wp-content/uploads/2021/01/… Right in the abstract, he states "The DRE metric measures the percentage of bugs or defects found and removed prior to delivery of the software. "
    – Alarias
    Mar 28 at 14:30

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.