I need to asses the efficiency/productivity of a project's testing process, which also uses SCRUM. How do I go about? Are there some peer-reviewed or well-known standard approachs to evaluate the testing process of a project? Are there any useful, industry standard metrics I can use?

  • 1
    Why do you need this? I have some ideas if you want to improve an apparent "QA bottleneck" (hint: It's often not actually a QA / test issue). May 7, 2014 at 17:13

1 Answer 1


It depends, and there are no industry standards.

Seriously. Any metric can be gamed (and will be, if you use it for assessment). I'm not aware of any standard approaches, not least because the teams are - or should be - evaluating themselves regularly and looking for ways to improve their own processes (if they aren't then they're probably using SCRUM-but... where they have the forms of scrum but are actually doing something not at all agile. This is sadly quite common and is a failure of management rather than a team failure).

Some of the things you can do:

  • Be part of the retrospectives. All of them, if possible. This is where the team evaluates its performance and looks to improve. If most of the problems that are highlighted in the retrospectives are external to the team, then any problems are with management, not the teams.
  • Discussions with the testers. Talk to the testers and ask them how they feel they're doing. If they don't think the information will be used as a stick in performance evaluations, they'll be honest (because I've yet to meet a tester who likes to be involved with something that's not meeting his/her standards).
  • If you absolutely must have metrics, there are exactly two that have any value. One is the ratio of customer-reported issues with a project to tester-reported issues. After you filter off the customer issues that relate to misunderstandings, the customer realizing after using the product that what they thought would work doesn't, and so forth (which happens but has nothing to do with how effectively your testers are working), you'll have a reasonable metric. What you should find is that the majority of customer-reported issues are edge cases (yes, you do have to do a fair bit of evaluation to use this information properly). The other metric is the WTF per minute metric. I'm actually not joking here - if the reaction of someone using the software for the first time is, effectively, "WTF were they thinking?" there is a big problem. The more often this reaction comes up, the bigger the problem.
  • No matter what you choose to do, do not blame your test team. I can't emphasize this enough. With very few exceptions, your team is doing the best they can with the information and constraints they've been given. They'll work with you if they know the goal is to build the best software possible. They won't if they think you're going to penalize them because things didn't go well. This is human nature - you will likely find that if you treat the evaluation as a way to get better software without blaming anyone your test team will be harder on themselves than anyone else would.
  • Wow, amazing answer. I especially agree with the last point. The majority of testers that are good at their job are perfectionists and expect the most out of themselves. Give them the tools they need to do this.
    – Paul Muir
    Mar 24, 2014 at 14:23
  • I was looking for an answer which would provide guidelines/metrics based on some kind of framework, but this one was still useful.
    – user7253
    Nov 24, 2014 at 18:53
  • @user7253 - there tend not to be too many of those despite organizations trying to make them happen, mostly because every software project is different and has different demands and needs.
    – Kate Paulk
    Nov 25, 2014 at 12:01

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.