Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have seen some SQA groups use defect arrival or "discovery" rates in order to determine whether a piece of software is ready to be released. The idea is that the rate at which defects are uncovered will slow to an acceptable level.

What are these techniques? How is an "acceptable" defect arrival rate determined?

Edit: Daniel's answer gives a nice list of items. To elaborate on the question: what are the methodologies that allow setting of appropriate rates of decline / discovery of defects?

Edit 2: This link shows an example of the methodology. Has anyone used this or something like it?

share|improve this question
add comment

2 Answers

up vote 2 down vote accepted

There are lots of ways to do this; some of the data points I have used in the past are:

  • Build breakage: is the automated build compilation/test failure rate going up or down?
  • Assuming an Agile development model, are the user stories complete and have they been reviewed by QA + stakeholders
  • Has the code freeze been effective? Are developers still checking in new features? This usually means there is more testing to be done
  • Is there a lot of code merges going on at the end of the project? This tends to cause instability and could be a sign you need to do more targeted testing
  • Automated testing confidence. The better your automated test coverage, the more confident you can be when the code changes
  • Are there any blocker bugs left open?
  • Is the find rate of critical bugs increasing or decreasing?
share|improve this answer
    
+1 Thanks for the list! Some of the methodologies I've seen the output from seem to take these measures (or similar ones) and apply a statistical model to them. I was hopeful that someone had pointers to these methodologies. –  Peter K. May 11 '11 at 8:10
add comment

I don't think that there can be such a concept as an "acceptable" defect arrival rate, in the sense of a target number as you typically want to discover them as fast as possible. So, in english, you want to find as many bugs as you can.

The key way to read that metric is that you want a consistent arrival rate over time, i.e. you want the defect arrival trend to be fairly flat and never zero, say between 15-25 per day as opposed to 2 one day and 50 the next.

Once you have your bugs being discovered consistently, you want to look at the bug fix rate.

The simple answer is to average the number of bugs fixed per day, and divide the total number of bugs by the average. That is approximately how many days until you reach zero bugs. So, if you are fixing 5 bugs per day and you have 200 active bugs, the earliest that you will ship is in 40 working days time. If you want to ship sooner, you will need to stop adding features and focus on fixing more bugs.

The same information can be used in reverse to calculate a maximum allowable bug count. Say you only have 40 days until your desired ship date, and you are fixing 5 bugs per day as in the previous example. If you active bug count is over 200 today, you will probably miss your target. This number continuously decreases so in 2 weeks time, with 30 working days to go, your bug count should be at the 150 mark if you are going to hit your ship date.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.