Industry Average Bug Fix Cost

Question

You often hear statistics like "$5,000 is the average cost of repairing a defect in production". While I understand that this varies greatly based on the company you work for, how they manage bug fixes and the salaries of those involved among other things, I'm being asked to find some kind of average. Anyone know where I might find a study or analysis of such things?

Answer 1

This may or may not help you, but my guess is that software bug cost is likely power-law distributed. This roughly means that there are many bugs that don't cost very much to fix (imagine a bug found from unit test results early in production code; the cost to fix that is essentially zero and there may be several of these kinds of bugs) and that there are a very small number of bugs that have a very high cost to fix (eg round-off errors causing rockets to explode). Quantities that are distributed with power-laws don't have well-defined mean values. Median values are a better quantity in these cases since they're better defined, but I'm still not sure what calculating such as value actually tells you about costs.

I don't have any hard evidence on hand for this, but I would be very careful when looking for "average cost per bug" or similar quantities, even with a narrow context.

Answer 2

"$5,000 is the average cost of repairing a defect in production"

I've never heard that. Do you have a source?

Unfortunately, I don't know of any meaningful Industry Average here. I'd also be very skeptical of anything that would purport to be one. I do know that many of these sorts of popular truisms "it costs 10 times as much to fix a bug in production as it does before production" are just made up without any real data behind them.

Personally, I think you'd be far better off trying to capture the costs within your specific company.

This might help: http://www.stickyminds.com/sitewide.asp?Function=WEEKLYCOLUMN&ObjectId=3223

"I'm being asked to find some kind of average"

Interesting. What is the motivation behinds such a question? What are they really after?

Answer 3

The Leprecahauns of S/W engineering book (http://leanpub.com/leprechauns ) has a chapter on 'The cost of defects: an illustrated history' where he tries to find the data behind the claim that the later a defect is found the more expensive to fix. Might be worth you reading the book to see how a lot of industry myths are exposed

Answer 4

It's common to see graphs in textbooks of the costs of fixing bugs found at various stages:

The source for the graph is Software Engineering Economics by Barry Boehm, which is probably worth getting hold of.

The cost of fixing a bug is a metric which probably has little value on its own. This article (Word doc) explains some of the problems:

The cost-per-defect-metric has developed into an urban legend, with hundreds of assertions in the literature that early defect detection and removal is cheaper than late defect detection and removal by more than 10 to 1. This is true mathematically, but there is a problem with the cost per defect calculations that will be discussed in the article. As will be shown, cost per defect is always cheapest where the greatest numbers of defects are found. As quality improves, cost per defect gets higher until zero defects are encountered, where the cost per defect metric goes to infinity.

More importantly the cost-per-defect metric tends to ignore the major economic value of improved quality: shorter development schedules and reduced development costs outside of explicit defect repairs.