What is the industry standard value for Defect Density in the Agile process? This is defined as the number of defects per thousand lines of code.
Is there any other way we can calculate the ratio? If so, please share the details/formula.
Add a comment
|
3 Answers
In my 20+ years in software, I've never come across a team/leader/engineer that cares about defect density as per this definition (bugs per line of code)! This measurement tends to be very old-school, pre-agile metric. I wouldn't even call it "industry standard" anymore.
With Agile, especially when using Jira or like software, people may care more about the number of bug tickets that are open or how long they have been open in the backlog. I say "may care more" because a lot of people just shrug at bugs in the backlog or say "it's just part of the process," or some other negative feeling statement. I usually have to fight for bugs to be prioritized more than anything else to get the team to care about fixing them! Again, this is team dependent.
Bugs/defects that are found during the sprint usually don't have an associated bug ticket (though this depends on the team). I usually document the bug in the feature story as a comment and pass it back to the developer. In this case, you can't really get a measure of the "number of bugs" since it's just a comment in the story. This type of documentation is more about preventing bugs from being released to production whereas backlog bugs are bugs/defects found in production or outside a specific story.
If you're looking for a percentage, you can take the "number of open bugs/number of open stories" in the backlog. For example, you have 300 bugs and 584 open stories = 51% bugs! As bugs are fixed or closed, hopefully, this percentage goes down. Now you can calculate closed bugs over time.
All this to say, the standardization of measuring defects has pretty much gone away other than a Jira report! Alternatively, especially with Agile, the use of DORA metrics is gaining more momentum! Not aware of DORA? Check out the book "Accelerate" by Dr. Nicole Forsgren
Because DORA is gaining popularity, you can also set up a service called Haystack! This app will automatically calculate your metrics on a merge request/pull request basis, by team member, by repository, etc. (Note, I have no affiliation with Haystack; I've just used it on prior teams).
Defect density is always defects per size. It doesn't necessarily need to be lines of code, though. I've also seen it expressed as defects per function point, but function point counting is not quite as straight forward as counting lines of code. However, it does acknowledge that some languages can do more per line of code, which makes it easier to compare across languages.
I'm not sure how many teams are tracking defect density, but I am aware of a publication titled "100 to 1 Ratio for Agile Defect Prevention Over Traditional Methods" by Nancy Van Schooenderwoert. It has a table that cites some data from Capers Jones which puts best and average defects per function point for non-agile projects somewhere between 2.0 and 4.5, while the author ran a project at 0.22 defects per function point and articles in the Cutter IT Journal found agile projects with fewer than 0.05 defects per function point.
Now, this doesn't necessarily mean that agile projects will inherently have fewer defects. Some agile practices - test-driven development, acceptance test-driven development, continuous integration, and specification by example, to give a few examples - do improve quality. However, the iterative and incremental nature of the software and the close collaboration with key stakeholders can keep the scope smaller and focused, eliminating waste in unnecessary features - after all, a defect in an unwanted or unnecessary feature is still a defect. Stakeholder collaboration can also find defects earlier. Agile does make sure that, at every small step, the product is acceptable to customers and evolves over time without big inspection and test activities at the very end to get shortchanged by schedule pressures.
Such metric doesn't have much sense, because working software is the primary measure of progress for agile projects.
With coding practices such as continuous integration and TDD, the developer's understanding of the solution is always implemented (otherwise you will have a failed test).
In this context, a realised defect can be only a misunderstand of the problem itself, which agile project tackle by welcoming changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage..
These changes are enabled through practices such as continuous delivery (keeping the software always in a deliverable state).
For truly agile projects, there are only Change of Requirements, not accumulated defects that you manage.
