0

I need some good statistics on how much effort do people spend on an average software bug. We assume that there is some interface between users and developers and wish to calculate just the bug costs, not the followup costs.

  • How long does it take an average user to report an average bug?
  • How long does it take an average developer to fix an average bug?

I need real numbers, with or without error bars. These may be given in hours or in currency such as USD or EUR, if you feel it more appropriate.

We wish to count only the time the user/developer actively do something useful, such as asking for more information and supplying it, saying thanks, coding, etc., but not just sit-and-wait times.

Good literature references and good analyis is welcome. I know that good numbers are difficult to come up with, that error margins can be large, an that the answer is industry-dependent, but please no bullshit. Of course, the terms "user", "developer", "bug", "average", "report", "fix", "industry" ... are imprecise; you may make these terms precise if you wish to.

closed as too broad by Peter M., Kate Paulk, Milin Patel, Bharat Mane, Tarun May 11 '16 at 8:22

Please edit the question to limit it to a specific problem with enough detail to identify an adequate answer. Avoid asking multiple distinct questions at once. See the How to Ask page for help clarifying this question. If this question can be reworded to fit the rules in the help center, please edit the question.

  • 1
    Give us some background. What is the goal of this exercise? How do you intend to use any resulting number? What is your environment/industry? My guess would be that effort to fix bug in Mars lander would be orders of magnitude higher compared to a senior project, so how you plan to compare results? – Peter M. May 9 '16 at 18:18
  • What would you consider as an average bug? How do you decide it is average? – Milin Patel May 10 '16 at 4:11
  • I've seen excellent graph for situation like yours; average performance of selected colors in in Excel pie graphs in last quarter in MS Office users in USA. It averaged numbers, and made no sense. Exactly like yours. :-) Or another one: average weight of mammal. – Peter M. May 10 '16 at 14:11
  • 1
    The numbers will be extremely skewed and nearly completely made up. Why Skewed? By saying all fields you have to account for things like bug fixes that require new hardware in places like the ISS. This will skew the results massively because if you account for the cost of a rocket launch into the cost of the bug than you're looking at a massively large number. Not to mention the amount of effort NASA goes through to ensure that there is no defects. So my answer will be ~$1 million per bug with about 2 months of labor, give or take a few million and a couple months of labor. – Paul Muir May 10 '16 at 16:28
2

There is no such thing as an average bug. Every bug is - by definition - an exception.

Furthermore:

  • Each industry segment has different expectations of quality. The expected standard for software powering medical devices is very different than the expected standard for casual game, to give just one example.
  • Within an industry, each organization has different expectations of quality.
  • Within a single piece of software, the prioritization of a bug can differ depending on where it is in the software, how it impacts the user experience and a number of other factors.
  • Even when two bugs are superficially very similar, the effort to fix them can be massively different depending on application architecture and many other factors.

In short, the only numbers that make any sense are aggregate numbers giving approximate team velocity, and even those only make sense for that team with that product.

  • @MarkMcGregor - sure, you can calculate the effort, but the number is meaningless. It can't be used for prediction unless you slice it by industry and organization, which leaves you right back where you started. It can't be used for cost calculations for the same reason. – Kate Paulk May 11 '16 at 11:29
0

This apparently looks like a case study to analyze things. Lets take both questions one by one.

How long does it take an average user to report an average bug?

Well, the straight forward bugs that are very apparent won't take a lot of time. It hardly takes any time to figure out if the home page of a website isn't opening. The bugs that take time and have importance are the ones that are scenario based. This is where imagination of a tester and testing skill comes into picture. These bugs definitely have more importance and takes longer in terms of time and money to speculate them.

How long does it take an average developer to fix an average bug?

This depends on the severity of bug. Straightforward bugs are easy to fix and its is again the scenario based bugs that require more time and investment. As the rule of thumb goes the bugs found earlier in development life-cycle are easier and more importantly cheaper to fix. The cost of fixing a bug increases as we move along the development life-cycle.

Thanks!

0

Although we're able to help in how to provide an estimate for time/cost and give some pointers as to what can affect this, it'll be close to impossible to an accurate estimate specifically for your scenario. Not because we don't want to, but because there are so many variables at play. On top of what everyone else has said:

  • An 'average' defect is different between companies.
  • The QA's ability to find and raise defects will vary.
  • The cost of resources will vary as well; a junior dev is cheaper than a senior dev, for example
  • Does it require multiple resources to find/fix the defect?
  • Average UX defects is cheaper (and less time-consuming) to fix than a back-and/calculations defect.
  • Fixing a defect may be worth $100 to a retail company, but $1000 to a finance company, so the industry you work in will impact your estimate

As Rohan pointed out, it's well documented that the later a bug is found the more expensive it is to fix (see this article on what is the costs of defects in Software Testing), as a defect in production can, potentially, have the further fallout of losing revenue through lost customers... which would have been avoided if found during system testing.

If you're able to make your question a bit more specific, we'll be happy to help in giving a more accurate esimation :)

Hope this helps!

0

This greatly depends per domain, in some domains defects are easy to record and easy to fix. In some it maybe extremely complex and time consuming to even describe the reproduction steps of a defect...

The best way to find out is to measure with a defect tracker (like Jira) where you can also log the amount of time spend on a issue. Then after a year you can report the averages for that domain.