It is said that increase in the appraisal costs (i.e. costs spent for finding non-corfomances - testing, reviews) decreases the failure costs (that comprise of internal (fixing of bugs we found + retesting ) and external failure costs (fixing bugs that customer found + retesting).

Well, I think it is only true if the cost for defects found by customer is higher. What I mean is:

when I invest in the testing, the cost of rework of defects it discovers will be more or less the same as if the customer discovered it. My developers are paid the same, no matter whether they fix a bug found by the customer or ourselves. Of course, there will be some overhead included and I do not take into account indirect costs like our reputation going down for producing buggy software.

But I just do not believe the failure costs decrease as appraisal costs increase, as several books state.
Take the following situations:

More testing, less customer defects Cost of testing: 10K (several issuses found)
Rework cost of internally found defects: 5K
Rework cost of externally found defects 3K
Total=18K.

Less testing, more customer defects
Cost of testing: 2K (not many issues found)
Rework of issues (few ours + all from customer): 12K. (more twice as much as in the previous situation)
Total=14K

What I saved on testing I spent on fixing and even saved a bit.

  • Where did you take those values from? – Alexey R. Oct 11 at 9:39
  • btw Rework of issues (few ours + all from customer): 12K. (more twice as much as in the previous situation) is actually 1.5 more (12 vs 5+3) – Alexey R. Oct 11 at 9:42
  • 1
    The cost of your customers finding defects isn't tangible, as you have to consider things like reputable damage which is often irreparable. For example, some people (like myself) won't make a purchase from a website littered with typo's.Or the software might be unusable because of the amount of bugs - look at Netscape Navigator. – trashpanda Oct 11 at 10:26
up vote 0 down vote accepted

You're missing several cost factors in your analysis. Some of them are external to your department but impact the company as a whole. Others are not.

Some of the missing cost factors:

  • Cost of customer support - Each bug that escapes to the wild will cause some amount of customer support time: the support call/email, time for the support person to debug and reproduce the problem, time to write up the problem and send to your group, then when you've fixed it and released the fix, time to verify it and notify the customer.
  • Cost of deployment - unless you are running a continuous integration and continuous test environment, there is a cost to building and releasing each deployment. If you are spending less time and effort catching the bugs earlier, you will be spending more time and effort making patch deployments. This leaves you with less time for new development.
  • Reduced conversion cost - It might be difficult to calculate, but the more bugs in a released product, the more likely a customer will decide that it's worth the effort to find an alternative.
  • Increased customer loss - How much this impacts you - and how the impact will manifest - will depend on the nature of your business: if you derive a significant portion of your income from long-term subscriptions to a service, you could lose a lot when customers move to an equivalent service. On the other hand, if you rely on sales, you could see sales numbers decreasing as word goes out about buggy products.
  • Increased context switching time - Bugs caught while development is in process or shortly after completion involve code that the developer is still familiar with. Bugs caught later, after the developer has moved on to other projects, require the person fixing them to refamiliarize themselves with the code and the business logic before they can fix the problem. They may also need to create new branches, shelve their current work in order to use the correct codebase, or any number of other setup activities. All of this takes time which can't be used to work on new features.

It's still possible to work out the full cost of allowing bugs to be released, but it's more difficult than you've listed.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.