13

Is there a place for gamification in software testing? Could it be made as a part of a regular testing process and could it have a positive impact on productivity and quality?


I've been thinking about gamifying our manual testing efforts, make up some "competitions" with badges and awards, here are some of the ideas:

  • number of bugs per a testing marathon - compete in a number of discovered bugs in a limited amount of time (say, per a testing sprint iteration). Bugs may have severity coefficients to have a weighted number of points per bug
  • compete in a funniest/weirdest bug per release and have a maintained "hall of fame"
  • set up an "artificial bug" scenario, when a tester is given a type of an existing problem (for instance, SQL injection vulnerability because of not properly validated input), but not given a place and steps to reproduce

What other ideas do you have in mind?

Please let me know if it simply sounds silly, counter-productive and will never work :)

  • 3
    Reproduce and narrow down intermitten legacy bug. A dragon that many have failed to kill. – dzieciou Jul 11 '17 at 7:37
  • 1
    @dzieciou, good idea, but from my own experiences, intermittent legacy bugs are normally closed after they cannot be produced for a while; if that happens, they are lost forever. – Yu Zhang Jul 11 '17 at 7:40
  • 8
    Relevant Dilbert: I'm gonna write me a new minivan this afternoon. – Nzall Jul 11 '17 at 11:34
  • 1
    Please don't make gamification part of the regular work - IMHO you would be dragging down the profession. A game night near the end of the release when the majority of work is complete - okay. Badges, awards, points as part of the regular work - gets old extremely quickly and cheapens what should be valuable professional work. – Joe Strazzere Jul 11 '17 at 20:20
  • 1
    You may enjoy reading "Bringing Out the Best in People" by Aubrey C. Daniels. It's about using positive reinforcement to increase the behaviors that you find desirable, which is basically what you're talking about. It can be incredibly effective if done correctly, but there are a surprisingly number of possible pitfalls. – Kat Jul 11 '17 at 20:22
9
+50

I am in for this idea. This can be a part of regular testing. But whether it will have a positive impact on productivity; is largely dependent on the way you implement this idea.

Definitely, there are positive aspects of gamification which you have talked about above.

I just want to focus on implementation part of it. Because we have to take care of the down side of it too.:

  1. Competition might lead to ego clashes between individuals which is counterproductive for the team.
  2. In order to get recognition, some team members might get into this race of raising bugs where even sometimes they are reporting crap.
  3. People might raise duplicate bugs and then again argument around who's bug is going to be marked as duplicate and closed.
  4. A team member, who has not been recognized even once during last one or two sprints, might get demoralized. So, we have to take care of that too.

But, having said that, I am not just against it. So, I want to add a few points to your list:

I think that the intent should be to recognize as many people as possible because it will be motivating for all.

  1. For example, recognition for the team members who reported Top 5 bugs (which were most important).

I am suggesting this award because from my experience, I have worked with people who are not very fast. They take their own time to test a functionality but are very thorough. So, the bugs that they log are of great importance to the project. So, this award will cover such people.

  1. Defect leakage: Another award to the person or team who tested a particular functionality and no defect ( or 1 or 2 low priority defects) was reported in that functionality in UAT.

  2. Review effectiveness award: This will ensure better reviews. You can set a target for that. I mean if the review effectiveness is greater than 30% (for example) then that reviewer will get the recognition (if more than 30% of the total bugs raised in a functionality were caught during reviews).

  3. No invalid defects: This one will improve the quality of the bugs reported. Gradually, if all the team members end up getting this and are not reporting anything that is invalid, I would be happy to give this to everyone.

  4. Apart from this, there can be team events too. You can set some measures at team level as well. For example, if team delivers a release that adheres to certain quality parameters then there will be some kind of treat/outing or something for the whole team (We have targets for quality matrices like Defect Removal efficiency, Defect Leakage and Review effectiveness) . This is important to keep the team spirit alive. Otherwise, I personally believe that too many individual awards/recognition can adversely impact the team spirit.So, we should maintain a balance between the two.

  • 1
    Very comprehensive; one speculation from me; in terms of avoiding ego clashing and demoralization, how about we reward people in private? But this may take away of the benefit of public recognition. – Yu Zhang Jul 11 '17 at 7:43
  • 1
    Well. Yes, I agree with you that people like their strengths to be talked about in public and any weaknesses; in private. So, people like to be recognized publicly. Suppose, there are people in your team who are "Very good" performers and add value to the team but they are not "Outstanding" performers. So, if they are not getting awarded for one or two sprints/releases then you can add an award like "Silent Killer" or "Consistent performer" or "Hidden Treasure" etc. – Aalok Jul 11 '17 at 8:35
  • 1
    But, ego issue; is difficult to deal with. In such cases, for every recognition, you can add a mandatory criteria and that is "Soft Skills". You can set some criteria like if a person was involved in a conflict, then 1 or 2 points will be deducted from soft skills rating and anyone having less than "X" points in soft skills won't be eligible for any recognition. I mean this can be worked upon in further detail. So, people will try not to get into any kind of conversation that can impact soft skills. Just a thought... – Aalok Jul 11 '17 at 8:38
  • 4
    @YuZhang Rewarding people in private doesn't help. You still know you didn't get the award. And are you suggesting that the team members hide their accomplishments from other people on the team? That sounds like a pretty bad way to work in a team :) – Luaan Jul 11 '17 at 9:39
  • 2
    Some people love their accomplishments to be made public, but some hate it. A possible alternative is to simply list the top five bugs without saying who found them. People who want it to be made public will brag; those who don't can smile to themselves in silent satisfaction. – Kat Jul 11 '17 at 20:17
4

As they say be careful what you measure - you will get it.

There are many questions in workplace forums about metrics and motivation proposals which went wrong, debating how different metrics can be misused and gamed.

You use "gamification" as having more fun with the task. "Gaming" is also used in opposite sense: using the metrics against the original intent.

Sadly, search by tags is broken for a while on SE, but try searching for "metrics" and "productivity". I know there are many relevant questions, it is just harder to find them with broken search.

Also, Murphy's law applies: if anything CAN go wrong, it WILL.

Few of the threads on Workplace Exchange about gaming (any) metrics:

  • Hello Peter, what do you mean by metrics can be misused? We have been using matrices for about last 4 years. A walk-through of the matrices is provided to the team after it is prepared after every sprint. Every team member is aware of how each and every calculation is done. Our manager is involved when it is prepared. Why would he dig a grave for himself by playing around with values or misusing it in anyway? – Aalok Jul 12 '17 at 2:54
  • 1
    @Aalok - you never seen any metrics disused? You must be new around the block? :-) Check this Dilbert cartoon from comments on OP. BTW, manager preparing the metrics is not a guarantee it will work - see the cartoon, idea by PHB. I am also not sure why you use "metrics" and "matrices" as they are the same. – Peter M. - stands for Monica Jul 12 '17 at 13:49
  • Yes Peter. You got it right. I am new around the block. But, interaction with experienced people like you will not keep me new for a long time. Cheers! In between, I admire you a lot. Have read many answers from you. Quite convincing..! – Aalok Jul 12 '17 at 16:50
3

I've seen this done successfully for an external beta program that used hundreds of enthusiastic users of the current product to test out the next version.

Points were awarded for simply using the product for n minutes each day, for being the first to report a bug (more points for higher severity bugs). We also would embed "prize words" in the UI of under-tested features, and award points to testers who discovered them.

At the end of the beta cycle, points could be redeemed for swag like t-shirts, coffee mugs, free software, etc. Testers with higher scores were invited to participate again in the next beta cycle.

This was long ago, so it required a hoard of beta coordinators to answer questions for beta testers, triage their bug reports, etc. Today, a lot of that could be automated.

  • I agree that this might be good for an external beta program - where you don't want to actually pay someone for professional QA work. I wouldn't do the same for actual employees. – Joe Strazzere Jul 11 '17 at 20:23
3

There's a lot of horror stories that come right out efforts to increase bug reports purely for the sake of having more of them. It can create a lot of animosity between developers and qa engineers. The "weirdest bug hall of fame" can be a mark of shame for the developer who released it into the software. When I do manual testing, I'd much rather just hear a "thank you".

I would suggest gamifying the training efforts, and building out a set of automated tests that will free manual qa engineers from having to test the same set of code repeatedly. This will allow them to focus on analysis and exploratory testing.

3

I'd be very cautious around this. If the company culture is right and the team is open, transparent, and very trusting, this could be an excellent and fun activity. Knowing this, adding any "metrics" whether they be aiming for fun or serious tracking, can have a significant impact on the team's mentality.

I would definitely shy away from the number of bugs, as this encourages people to raise bugs as much as they can, and by the fact that you use the term "bugs" you are already implying you have a negative view of "identifying work not yet done" within the team.

As an aside, this question raises some questions around your development methodologies. You mention sprints so I am going to assume you are using Scrum or some variation, in which case there shouldn't be "bugs" during a sprint - merely tasks or pieces of work that the team hasn't done yet. It's positive to identify these things earlier rather than later, whether a developer or tester identifies it. Your language seems to be driving distance between "dev" and "test", encouraging mini-waterfall rather than agile practices. Whatever "game" you create should equally apply to devs as testers, as QA is a whole team responsibility in agile teams.

I might be reading too deeply into this, but the terminology you have used makes me think this gamification won't be a positive move for the team.

2

Gamification is the act of applying game-like characteristics to a non-game situation in order to increase ‘user’ interaction.

An informal survey ran amongst my gaming (computer, board or roleplay) friends returned a nearly unanimous verdict that the returns they value from play-testing or beta-testing are the kudos of having been involved, early access to the game and the feel-good factor when they see the product changing based on their feedback. On a lesser note, they also appreciated it if their name appeared in the credits of the game on release.Finding challenges is easy, but finding resolutions for them is hard. It is difficult to look at the full scope of testing, instead it might be better to start with one part. In a recent meetup focused on gamification in testing, we discussed what part to select and agreed that regression testing could be a valid part to start at. Everyone had ideas on how to make it more motivating. In many cases the discussion was about that the participants did not want to do regression testing in a way that was meaningless. The group identified several things that could be done to avoid doing tasks in a meaningless way. We considered how to add gamification mechanics to make it more motivating. One reflection that we had was that a lot of what we thought we could gamify had to do with feedback to others or from others, information sharing and basically about improving communication. My own reflection from the meetup was that it was possible to use gamification to communicate value in testing with people that might have different ideas on what testing was.

  1. It is a cost effective way to increase product quality
  2. It motivates your team to produce better quality work Software development and testing can be boring if we repeat the same routine again and again. So why not to make a fun game after coding complete? In order to make a test campaign fun, you need to be fair. You need to define the weight of different kind of bug differently. You need to set ranking for the different level (For example Godlike, Knight, Grown-Up, Baby), as a goal for participants to aim. You need to keep everyone updated on the status by publishing the result daily. At the end of the campaign, you need to reward the top performer publicly and make it a carnival. It can be fun and build up morale within your company.

Gamification in software testing definitely is not a replacement for all other software testing options. However, it can bring unique values and be part of your software testing strategy in your organisation. It might be a lot of work to evaluate and calculate all the scoreboard result manually.

2

I am both for and against this !!!

I am against using it with factors such as:

  • most number of bugs found
  • highest number of end-to-end tests written
  • most number of bugs closed this month

We've learned over time that it can be easy to fall into the trap of focusing on the measurement, not the outcome. This can lead to situations such as filing more and more bugs that are not relevant to the business success.

I am for it when it is considered against factors such as:

  • more help given to others
  • lower frequency of intermittent failures
  • higher test coverage
2

I've tried similar gamification ideas in the past, though in helpdesk teams. Here's some experiences:

  • Some of the team took to it and some didn't - so the team members that were not interested became more withdrawn and weren't involved in the inevitable banter among the enthusiasts: not good for the team
  • Enthusiasts created more tickets (duplicates/variations that should have been extensions of previous tickets): Testers will potentially log multiple bug variations
  • The enthusiasts didn't want to work with other team members/discuss an issue in case credit for the solution was stolen. I can see this happening in testing: Testers won't discuss issues together and conclude based on collective intelligence - they'll just log bugs
  • Team members with specialisms in certain areas would previously be assigned tickets, but others wanted to keep them for their score: Issues were not solved efficiently so not good for the customer

A couple of other thoughts:

  • Testers working on more complex software will score more points
  • Testers will want to work with lower quality developers!

It might work better with testers but there will be inevitable changes in team behaviour that won't always be positive =/

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.