Gamification in software testing

Question

Is there a place for gamification in software testing? Could it be made as a part of a regular testing process and could it have a positive impact on productivity and quality?

---

I've been thinking about gamifying our manual testing efforts, make up some "competitions" with badges and awards, here are some of the ideas:

• number of bugs per a testing marathon - compete in a number of discovered bugs in a limited amount of time (say, per a testing sprint iteration). Bugs may have severity coefficients to have a weighted number of points per bug
• compete in a funniest/weirdest bug per release and have a maintained "hall of fame"
• set up an "artificial bug" scenario, when a tester is given a type of an existing problem (for instance, SQL injection vulnerability because of not properly validated input), but not given a place and steps to reproduce

What other ideas do you have in mind?

^{Please let me know if it simply sounds silly, counter-productive and will never work :)}

Answer 1

I am in for this idea. This can be a part of regular testing. But whether it will have a positive impact on productivity; is largely dependent on the way you implement this idea.

Definitely, there are positive aspects of gamification which you have talked about above.

I just want to focus on implementation part of it. Because we have to take care of the down side of it too.:

1. Competition might lead to ego clashes between individuals which is counterproductive for the team.
2. In order to get recognition, some team members might get into this race of raising bugs where even sometimes they are reporting crap.
3. People might raise duplicate bugs and then again argument around who's bug is going to be marked as duplicate and closed.
4. A team member, who has not been recognized even once during last one or two sprints, might get demoralized. So, we have to take care of that too.

But, having said that, I am not just against it. So, I want to add a few points to your list:

I think that the intent should be to recognize as many people as possible because it will be motivating for all.

1. For example, recognition for the team members who reported Top 5 bugs (which were most important).

I am suggesting this award because from my experience, I have worked with people who are not very fast. They take their own time to test a functionality but are very thorough. So, the bugs that they log are of great importance to the project. So, this award will cover such people.

2. Defect leakage: Another award to the person or team who tested a particular functionality and no defect ( or 1 or 2 low priority defects) was reported in that functionality in UAT.

3. Review effectiveness award: This will ensure better reviews. You can set a target for that. I mean if the review effectiveness is greater than 30% (for example) then that reviewer will get the recognition (if more than 30% of the total bugs raised in a functionality were caught during reviews).

4. No invalid defects: This one will improve the quality of the bugs reported. Gradually, if all the team members end up getting this and are not reporting anything that is invalid, I would be happy to give this to everyone.

5. Apart from this, there can be team events too. You can set some measures at team level as well. For example, if team delivers a release that adheres to certain quality parameters then there will be some kind of treat/outing or something for the whole team (We have targets for quality matrices like Defect Removal efficiency, Defect Leakage and Review effectiveness) . This is important to keep the team spirit alive. Otherwise, I personally believe that too many individual awards/recognition can adversely impact the team spirit.So, we should maintain a balance between the two.

Answer 2

As they say be careful what you measure - you will get it.

There are many questions in workplace forums about metrics and motivation proposals which went wrong, debating how different metrics can be misused and gamed.

You use "gamification" as having more fun with the task. "Gaming" is also used in opposite sense: using the metrics against the original intent.

Sadly, search by tags is broken for a while on SE, but try searching for "metrics" and "productivity". I know there are many relevant questions, it is just harder to find them with broken search.

Also, Murphy's law applies: if anything CAN go wrong, it WILL.

Few of the threads on Workplace Exchange about gaming (any) metrics:

• https://workplace.stackexchange.com/questions/69894/how-to-handle-a-test-engineer-biased-to-one-of-our-team-member
• https://workplace.stackexchange.com/questions/5940/how-to-measure-the-creative-workers-output-who-are-not-filling-in-the-timesheet
• https://workplace.stackexchange.com/questions/87887/how-to-objectively-measure-colleague-cooperation (mentions: http://cuberules.com/2008/02/07/management-by-numbers-be-very-afraid/)
• https://workplace.stackexchange.com/questions/22087/measure-competence-of-the-company-on-different-technology-areas
• https://workplace.stackexchange.com/questions/48053/how-productivity-or-working-hours-are-measured-in-software-development-jobs

Answer 3

I've seen this done successfully for an external beta program that used hundreds of enthusiastic users of the current product to test out the next version.

Points were awarded for simply using the product for n minutes each day, for being the first to report a bug (more points for higher severity bugs). We also would embed "prize words" in the UI of under-tested features, and award points to testers who discovered them.

At the end of the beta cycle, points could be redeemed for swag like t-shirts, coffee mugs, free software, etc. Testers with higher scores were invited to participate again in the next beta cycle.

This was long ago, so it required a hoard of beta coordinators to answer questions for beta testers, triage their bug reports, etc. Today, a lot of that could be automated.

Answer 4

There's a lot of horror stories that come right out efforts to increase bug reports purely for the sake of having more of them. It can create a lot of animosity between developers and qa engineers. The "weirdest bug hall of fame" can be a mark of shame for the developer who released it into the software. When I do manual testing, I'd much rather just hear a "thank you".

I would suggest gamifying the training efforts, and building out a set of automated tests that will free manual qa engineers from having to test the same set of code repeatedly. This will allow them to focus on analysis and exploratory testing.

Answer 5

I'd be very cautious around this. If the company culture is right and the team is open, transparent, and very trusting, this could be an excellent and fun activity. Knowing this, adding any "metrics" whether they be aiming for fun or serious tracking, can have a significant impact on the team's mentality.

I would definitely shy away from the number of bugs, as this encourages people to raise bugs as much as they can, and by the fact that you use the term "bugs" you are already implying you have a negative view of "identifying work not yet done" within the team.

As an aside, this question raises some questions around your development methodologies. You mention sprints so I am going to assume you are using Scrum or some variation, in which case there shouldn't be "bugs" during a sprint - merely tasks or pieces of work that the team hasn't done yet. It's positive to identify these things earlier rather than later, whether a developer or tester identifies it. Your language seems to be driving distance between "dev" and "test", encouraging mini-waterfall rather than agile practices. Whatever "game" you create should equally apply to devs as testers, as QA is a whole team responsibility in agile teams.

I might be reading too deeply into this, but the terminology you have used makes me think this gamification won't be a positive move for the team.

Answer 6

I've tried similar gamification ideas in the past, though in helpdesk teams. Here's some experiences:

• Some of the team took to it and some didn't - so the team members that were not interested became more withdrawn and weren't involved in the inevitable banter among the enthusiasts: not good for the team
• Enthusiasts created more tickets (duplicates/variations that should have been extensions of previous tickets): Testers will potentially log multiple bug variations
• The enthusiasts didn't want to work with other team members/discuss an issue in case credit for the solution was stolen. I can see this happening in testing: Testers won't discuss issues together and conclude based on collective intelligence - they'll just log bugs
• Team members with specialisms in certain areas would previously be assigned tickets, but others wanted to keep them for their score: Issues were not solved efficiently so not good for the customer

A couple of other thoughts:

• Testers working on more complex software will score more points
• Testers will want to work with lower quality developers!

It might work better with testers but there will be inevitable changes in team behaviour that won't always be positive =/

Answer 7

Gamification is the act of applying game-like characteristics to a non-game situation in order to increase ‘user’ interaction.

An informal survey ran amongst my gaming (computer, board or roleplay) friends returned a nearly unanimous verdict that the returns they value from play-testing or beta-testing are the kudos of having been involved, early access to the game and the feel-good factor when they see the product changing based on their feedback. On a lesser note, they also appreciated it if their name appeared in the credits of the game on release.Finding challenges is easy, but finding resolutions for them is hard. It is difficult to look at the full scope of testing, instead it might be better to start with one part. In a recent meetup focused on gamification in testing, we discussed what part to select and agreed that regression testing could be a valid part to start at. Everyone had ideas on how to make it more motivating. In many cases the discussion was about that the participants did not want to do regression testing in a way that was meaningless. The group identified several things that could be done to avoid doing tasks in a meaningless way. We considered how to add gamification mechanics to make it more motivating. One reflection that we had was that a lot of what we thought we could gamify had to do with feedback to others or from others, information sharing and basically about improving communication. My own reflection from the meetup was that it was possible to use gamification to communicate value in testing with people that might have different ideas on what testing was.

1. It is a cost effective way to increase product quality
2. It motivates your team to produce better quality work Software development and testing can be boring if we repeat the same routine again and again. So why not to make a fun game after coding complete? In order to make a test campaign fun, you need to be fair. You need to define the weight of different kind of bug differently. You need to set ranking for the different level (For example Godlike, Knight, Grown-Up, Baby), as a goal for participants to aim. You need to keep everyone updated on the status by publishing the result daily. At the end of the campaign, you need to reward the top performer publicly and make it a carnival. It can be fun and build up morale within your company.

Gamification in software testing definitely is not a replacement for all other software testing options. However, it can bring unique values and be part of your software testing strategy in your organisation. It might be a lot of work to evaluate and calculate all the scoreboard result manually.

Answer 8

I am both for and against this !!!

I am against using it with factors such as:

• most number of bugs found
• highest number of end-to-end tests written
• most number of bugs closed this month

We've learned over time that it can be easy to fall into the trap of focusing on the measurement, not the outcome. This can lead to situations such as filing more and more bugs that are not relevant to the business success.

I am for it when it is considered against factors such as:

• more help given to others
• lower frequency of intermittent failures
• higher test coverage

Answer 9

I just loved the third idea you talked about. It would be really exciting to see how a manually created ‘artificial bug’ could be identified based on the tester’s skills and expertise. If I need to add something to the gamification of the software testing process, I would like to see the use of defect tracking, once the bug is identified and removed from the product under test. This is actually an important factor that all quality assurance testing services need to understand and work with their testers in order to yield more sustainable outcomes.