6

You often hear statistics like "$5,000 is the average cost of repairing a defect in production". While I understand that this varies greatly based on the company you work for, how they manage bug fixes and the salaries of those involved among other things, I'm being asked to find some kind of average. Anyone know where I might find a study or analysis of such things?

1
  • I can't really speak authoritively on this, but it seems like there are way too many unknowable variables to be able to calculate the cost of repairing a bug in production. Sure, there's the actual cost of removing the bug and rolling out the fix, but you also have to take into account the loss of customer confidence that could translate to lost business in the long run, and balance it against the potential loss of morale and productivity that could result if the dev team has to work long hours on things that "Should have been caught earlier..."
    – user867
    Mar 26, 2012 at 4:34

6 Answers 6

5

It's common to see graphs in textbooks of the costs of fixing bugs found at various stages:

Relative Cost to Fix an Error

The source for the graph is Software Engineering Economics by Barry Boehm, which is probably worth getting hold of.

The cost of fixing a bug is a metric which probably has little value on its own. This article (Word doc) explains some of the problems:

The cost-per-defect-metric has developed into an urban legend, with hundreds of assertions in the literature that early defect detection and removal is cheaper than late defect detection and removal by more than 10 to 1. This is true mathematically, but there is a problem with the cost per defect calculations that will be discussed in the article. As will be shown, cost per defect is always cheapest where the greatest numbers of defects are found. As quality improves, cost per defect gets higher until zero defects are encountered, where the cost per defect metric goes to infinity.

More importantly the cost-per-defect metric tends to ignore the major economic value of improved quality: shorter development schedules and reduced development costs outside of explicit defect repairs.

4

"$5,000 is the average cost of repairing a defect in production"

I've never heard that. Do you have a source?

Unfortunately, I don't know of any meaningful Industry Average here. I'd also be very skeptical of anything that would purport to be one. I do know that many of these sorts of popular truisms "it costs 10 times as much to fix a bug in production as it does before production" are just made up without any real data behind them.

Personally, I think you'd be far better off trying to capture the costs within your specific company.

This might help: http://www.stickyminds.com/sitewide.asp?Function=WEEKLYCOLUMN&ObjectId=3223

"I'm being asked to find some kind of average"

Interesting. What is the motivation behinds such a question? What are they really after?

1
  • 1
    Agree with Joe - why are they asking and what will they do with the data if you could find it ? Mar 24, 2012 at 10:50
4

This may or may not help you, but my guess is that software bug cost is likely power-law distributed. This roughly means that there are many bugs that don't cost very much to fix (imagine a bug found from unit test results early in production code; the cost to fix that is essentially zero and there may be several of these kinds of bugs) and that there are a very small number of bugs that have a very high cost to fix (eg round-off errors causing rockets to explode). Quantities that are distributed with power-laws don't have well-defined mean values. Median values are a better quantity in these cases since they're better defined, but I'm still not sure what calculating such as value actually tells you about costs.

I don't have any hard evidence on hand for this, but I would be very careful when looking for "average cost per bug" or similar quantities, even with a narrow context.

0
1

The Leprecahauns of S/W engineering book (http://leanpub.com/leprechauns ) has a chapter on 'The cost of defects: an illustrated history' where he tries to find the data behind the claim that the later a defect is found the more expensive to fix. Might be worth you reading the book to see how a lot of industry myths are exposed

1

You can calculate the cost of defect leaked to production based on following factors:

Effort: Typical effort when a defect is found in prod is split across - analysis, reporting, reproduction of issue, fixing, deploying, testing(scripting, execution ,monitoring) reporting.

Resource cost: Onshore/Offshore (billing rate) Loss to business: application downtime, slowness, customer dissatisfaction, unable to do certain type of transaction, credit loss.

You can get notional value not real but it is good to conduct this exercise so that important of proper QA is determined.

0

Usually i use this simple formula: Cost to fix a defect = (Number of people * number of days) * cost per person-day

Best, Belal Raslan

1
  • 1
    This doesn't really take into account the different costs: for example if you have to fix a bug in production code the costs of a whole new release and deployment. Jul 22, 2014 at 10:36

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.