0

Does defect density make for a good KPI?

Is there any baseline which recommends that defect density should be above or below a certain amount for a release to be considered as 'healthy' or a 'better release'?

  • Are you asking if defect density is an effective KPI? – trashpanda May 2 at 7:37
  • Yes,And can some one please explain me why some one or two give me -2 points for this question? – ChathuD May 2 at 8:12
  • Voting to close as primarily opinion based. The community can not set your normal/base/standard. – Niels van Reijmersdal May 2 at 8:29
  • I think it could be a good question if it were framed less objectively. Whilst you're right that we can't set OP's baseline, we could offer advice as to why defect density doesn't make for an effective KPI... or maybe you disagree, and it does make for a good KPI? – trashpanda May 2 at 8:42
  • @NielsvanReijmersdal Voting to close ...mmm.. Do you really thing this is something we need to close.Then why dont people answer .Because people who need t close this Question must have the answer to it. – ChathuD May 2 at 8:51
2

As with many KPIs that try to use a quantity of defects as part of the measurement I'd say "Yes, but use it with caution. It's a matter of context".

As an example, most of my career has been with startups in varying stages of maturity. Often what we were developing wasn't meant to be entirely bug free, stable, and able to scale. It was meant to solicit feedback. In situations like that, an absolute expectation for density may not be useful and may not be how you want the business to measure the success of your team. Any reasonably experienced and capable test engineer can shake an immature applications and a lot of defects will fall out.

An alternative I'd suggest considering is approaching density as "how many defects did the internal team find and report" vs "how many did customers find". If applications are released and fixing know defects is deferred to later this is an intentional decision about how to use resources and not necessarily an indicator of success for development or QA. If an application is shipped and the customer finds all the bugs while the team is unaware of them I'd argue this is a riskier situation.

Additionally, I think density is a fantastic factor to consider when doing exercises like risk analysis which can be useful when determining how to allocate finite development and testing resources for future work.

  • Agreed. Defect detection rate (found pre-release vs post) is a better overall metric, but also benefits from breakdown by severity (impact x frequency) to round out the picture. Defect density, like all data, can be useful but is relative to the intent of the data consumer. Are you looking for problem areas in the codebase? Are you looking for problem areas in personnel? Static analysis tools are likely to provide more and better objective information than defect density, imo, but it still can't hurt depending on the situation. – Subjective Reality May 24 at 0:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.