Types of metrics used in QA management

Question

I've been software testing in and out of leadership roles for about 13 years now but the only really reporting or metrics I've done are basic bug reports, which, to be honest don't really show much.

I've been asked to show some type of metric that will somehow show something like 'is the team going better or worse?'. 'Is the team improving since last release?'. I was curious on how other companies, or yourself personally use which type of metrics to measure this.

Tools we currently use are JIRA, Xray (test cases), QF-Test (automation) & Cypress.io (automation). We also have BigPicture for JIRA but that's only used with upper management.

Thanks in advance for any suggestions or leading me on the right path to helping me get a couple metrics in place!

Answer 1

First of all, you need to define better what you want to measure. Imagine a sports team. One could say that "improving" means "winning more games" - and one could say that a software team is "improving" when the product is making more money or has a bigger user base. However, in both cases, one can break down what makes the team and analyse specific aspects. It will be highly contextual, depending on your strategy (both in the sport and on the software development).

One common important factor for testers is "are we being able to test more, or are we doing other things?". If that's important to you, you can look at Session-Based Test Management (SBMT).

The data input for SBMT will be the test charters:

From the charters, you can extract information. For instance, how much of the tester's time is spent in different activities:

This can identify problems with testability for instance.

Or how many testing sessions are done throughout time:

Of course, you can think of many other things: You can make a word cloud from the "Bugs" session, in order to identify things that more frequently happening on the bugs; how are you dividing your testing between OS/devices, etc...