How should a Software Tester deal with missed Defects/bugs in Production?

Question

I work as a QA/test engineer in a small software company. It is an e-commerce based product. I was assigned a few projects by my manager. I mostly do functional/blackbox testing. Before the production release, I tested many good scenarios and found quite a few defects in the code. Most of them got fixed. I did my analysis and gave important information to the stakeholders. I had a good test strategy at that time. Now, this module went live in production servers and all was good.

After many months, there were some enhancements in the product and I did regression testing of the same product, that old code build. Due to my carelessness, I now found at least 10 defects, bugs that were already in Production. I had missed those simple test cases. I should have found those earlier, many months ago. I'm no junior, I'm a senior analyst, which is shameful.

My manager and the stakeholders don't trust my work as a tester, as how dare I say that it PASS the QA the first time. They think that I'm careless, not worthy of working, and risky.

Now, I'm feeling very bad and low as one should be PERFECT in this job and there should be NO scope for mistakes. Company pays me to test well and not be careless. I feel I don't deserve this job and should find new job in another industry.

1. Should I ignore this mistake and keep on working in the same project?
2. Should I have acted more clever and wise. Should I have not found bugs in Production and should have been quiet? Why did I highlight my mistakes​ and been noticed by management.
3. Should I resign or change my Job, as I can't even do simple testing? I also broke the trust of management as well as my reputation.

Answer 1

Company pays me to test good and not be careless.

You are asking this question here - you are not careless, you care about the job you do and the things you can improve on.

Don't take it personally and be professional

The point of finding bugs is not to blame anyone, but to enhance the product's quality. If your manager is blaming you personally, this is an unhealthy and discouraging situation. It is counter-productive. Focus should be on what can be done to avoid this happening in the future, not who is responsible for what happened.

Perfect software is a myth

There will always be bugs - and there will always be bugs in production, it just happens.

This is an opportunity

I would not concentrate too much on the fact that you've missed some defects. Instead, look at this as an opportunity. Show your supervisors the things you've done to improve on quality control and testing:

• investigate what caused these bugs and defects, identify the problems - lack of certain tests, user stories, test cases, acceptance tests, load tests etc
• think of other places in your system where these kind of problems may occur - you may potentially discover similar problems in other parts of your product(s), which you may show and "claim". This may be a good sign for your supervisor(s) that you've learned your lesson and trying to improve the situation
• create automated test cases for these defects - make sure next time they are gonna be caught early and would not make it to production
• gather coverage and complexity reports - check if there are critical and complex parts of the system not covered with tests
• look into alternative testing methods, like Property-based testing as an alternative to common Example-based testing
• in case of manual testing, try to broaden the techniques you use, here are some follow-up resources:
  • Test Heuristics Cheat Sheet
  • What are some Problem solving techniques that can be used in testing?
  • How can a Software Tester use "Out of the Box" thinking approach to find more bugs?

Forget about your title - aim for "zero"

I should have found those earlier, many months ago. I'm no junior, I'm a senior analyst, which is shameful.

Don't let the pressure of being on a senior position have a negative effect on you. Aim for zero, listen, observe, offer and advice. Something I've learned from Chris Hadfield's "An Astronaut's Guide to Life on Earth":

In any given situation, according to Hadfield, you’re either a ‘plus-one’, a ‘zero’ or a ‘minus-one’. If you’re a plus-one, you’re actively adding value. If you’re a zero, you’re generally competent and don’t get in the way. Being a minus-one sucks, because you’re a liability and actively cause problems.

However, if you’re a plus-one and you walk into a situation trying to prove how great you are, you can go from a plus one to a minus one – your ‘I got this’ mentality might easily irritate and prove detrimental to the dynamic.

So the best thing to do in a new situation? Aim for zero. Listen. Observe. Offer advice. Don’t try to take control of everything. If you know what you’re doing, you won’t need to tell people you’re a plus one. They’ll know it.

Answer 2

Please do not resign.

As alecxe has said in his post, Perfect Software is a myth.

I would like to add, the difference between a tester and a developer is: It is very difficult to see how much a tester has contributed.

• When a developer is given a task to develop a feature, this developer will produce a tangible result, e.g. a clickable button.
• But when a tester is given a task to test a feature, it is very difficult to tell how much result we can get from this tester's work. Why? When something does wrong, it can go wrong in a billion way; given the finite amount of time and effort we are humanly allowed, it is not practical for us to catch all the bugs. Take a clickable button for an example, I have seen a clickable button failed to be clicked when it has been clicked exactly 56 times. Will this button fail at the 1000th click? How about 99999th click? We will never know.

Software testing's purpose is not to catch all the bugs, it is to provide confidence towards software quality within a limited amount of time and effort.

One suggestion that I want to make here is:

• Please keep a record of what you have done and how many bugs you have managed to catch, you can use the records as proof of due diligence should you need to.

Answer 3

As the other answers have said, do not blame yourself. Nobody can test any piece of software completely, any more than anyone can write completely bug-free software. It's too complex.

I'm going to quote an example from an article I wrote over at the Ministry of Testing Dojo: Ten Misconceptions About Software Testing. The quote is from a section explaining why bugs in production do not mean that testers failed.

It's silly to think testers should be able to catch everything that's wrong with a program. Take a basic calculator application, with or without scientific or other advanced functions. Now consider the addition function. To test only adding two integer numbers completely, you would have to test adding every number the calculator is capable of using to every number the calculator is capable of using and making sure that the result is correct and the display is correct.

Even with a very simple calculator that doesn't display exponential notation and can only add from -999999 to 999999, that's 1999998 possible integer numbers to add to 1999998 possible integer numbers - or (1999998)^2. If it takes an average of only 5 seconds to perform the calculation and check the result, you're still looking at over 300000 years to test every possible addition of two numbers (and that's working 24/7/365). Nobody has that much time.

Even if you automate and can bring that average test time down to 0.1 seconds, you're still looking at thousands of years, or thousands of computers, before you've tested every possible combination. And remember, this hasn't even touched anything beyond adding two integers. What happens when you add decimals? Other operations? More than two numbers? It doesn't take much for the number of possible outcomes to approach infinite.

(the main reason I'm quoting this is because it's easier than paraphrasing)

Now, consider this:

1. You found the problems in production. Not customers. That means they were still caught before they were "exposed" to external users, and it means that should any customers report them, the company can say "Yes, this is a known problem and we are working on a fix for a future release". That is not a failure.
2. You did the responsible thing and reported the problems as soon as you found them. By doing so, you limited the amount of risk your company is exposed to.
3. If your manager and stakeholders expect perfection, they will be disappointed. You did not cause the bugs to be in the code. It's entirely possible your developers didn't either: the bugs could well have been the result of unclear specifications or a miscommunication between the product owner and the developers. Or they could be what I call "stealth enhancements" where the product doesn't work quite the way customers expect so they report it as a bug even though they signed off on the use case documents.
4. The fact that you are beating yourself up over this is a sign of your dedication, not that you are a failure. A failure wouldn't care.
5. Are you Superman? God? Of course not. Expecting that you can catch every bug in software that has close to literally infinite possible user paths (particularly when you consider the ability to to undo/redo any number of times) is expecting yourself to be some kind of infallible being.

Convincing Your Manager To Support You

1. Stop Blaming Yourself Accept that yes, bugs slid through to production, but don't accept that this makes you unreliable. You did the best you could in the situation you had.
2. Show The Math Show your manager the math I used in the example - or even better, do a similar kind of example for your software. If you have say 10 independent true/false configuration flags, to test every function in your application with every possible configuration flag, you must test every function in your application 1024 (2^10) times. If it takes 5 hours to go through every function, that's over 5000 hours of testing. I rather doubt your manager wants you to do that, and I guarantee you don't want to do that. The tedium is mind-numbing.
3. Do Root Cause Analysis Not just on why you missed those particular bugs, but on how those bugs got in to the code base in the first place. The question "How can I be held responsible for code I did not write?" can be useful, as can referring back to the fact that you can't test everything.
4. Blame is Counterproductive You need to ask your manager why you are being blamed when you didn't introduce the bug to the code, when it would be much more productive to try to take measures to prevent similar bugs from being introduced to the code in future.

[Edited math in point 2 to correct error]

Answer 4

You are too hard on yourself.

Let me give you some insight on testing: you might wonder why testers even exist: if everyone would do a good job, testers would not be necessary.
Is this true? No, it is complete nonsense!
Why? Simply because a human has the tendency to overlook his/her own mistakes!

Now this fires back towards the tester him- or herself too: just as it's the job of the tester to test things the developers are not thinking about, it's up to the developers/project leader to pinpoint the areas the tester did not think about!

In other words, I advise you to do the following: start writing a test document, describing what you will test, which areas you will cover. The project leader needs to approve this document. This document can and will change during the course of the project, so just as a regular update is needed, so is a regular update review.

With this approach, if you didn't think of testing some areas, the burden won't be upon you completely, it will be a shared responsibility.

And while introducing this idea to your manager, you can say something like the following: "Dear management, the fact that I have missed some areas during the testing phase has put me in a very bad mood. After some downtime I have thought how I could turn this failure into an opportunity in order to avoid this in future and to be able to deliver better quality. Hereby my proposal: " (and you explain about the test document and its regular reviews).

Good luck

Answer 5

You and your manager seem to expect perfection from you. If perfection in employees were possible, you'd be out of a job, because they'd just hire perfect developers which don't introduce any defects.

It sounds like either you're starting on a depression - which is a medical condition that requires treatment - or management is abusing you. Or both. A QA pass doesn't mean there are no bugs, and every person knows that, including managers.

At this point I think this question can get more valuable answers on workplace.stackexchange.com

I suggest against approaches 1, 2, and 3 and instead suggest:

• Look into what depression is and consider visiting a mental health professional.
• Be honest and report bugs you find.
• Based on how you found the new defects, adjust your test plan.

Answer 6

Any employer who makes a tester (or a developer for that matter) feel bad about finding bugs deserves all its testers to leave and find a better employer. Everyone should feel good about finding bugs, whatever stage of the process you find them. If people feel guilty about finding bugs, they'll stop finding them, and that's the last thing you want.

Answer 7

The team should own the quality, not just the "tester." The team includes all involved parties (pm, devs, qa, team lead, even stakeholders).

The idea that the tester owns quality fosters apathy in developers and leads to siloed workflow where developer neglects even simple i/o testing and "throws it over the wall" for the tester to say it works.

Do not hang your head low. You did not create the bug, rather - you identified it.

As for whether you should resign... It sounds like the environment in which you work focuses heavily on blame rather than finding solutions. If that is your cup of tea, stay. If that is not the company culture that you thrive in, perhaps it is time to look for something more rewarding.

Quick additional note to ask yourself: am I truly senior level status while doubting my own ability?

Answer 8

Brother, every QA like you face similar challenges in early stage of the career. Initially everyone is careless and hence they do such things which is quite normal.

I'd advise to not to change the project or organization, because you have now lot much product knowledge.. Utilize it and gain trust of management by doing great job day by day.. Initiate something which is not done in your organization earlier. Try to automate test-cases which are error-prone.. Try to automate regression suite.. Send daily reports to your manager etc.. This will re-build trust for you..

Also from now make sure to understand full functionality, in scope and out of scope items, browsers, devices to be tested, FS fully understood, timelines etc..

If you take care of above activities, you will feel that there is no job like QA.

All the very best.

Answer 9

I don't want to repeat everything that's already been said but remember your ISTQB - Exhaustive Testing Is Impossible! Bugs will find their way into Production, and not even a testing team can stop that.

That brings me onto the point I wanted to make. It sounds like you are the sole tester on the project, and that is going to reduce scenario coverage just by human nature - nobody can think of every scenario by themselves. That's why testers like to come in teams, so we can check each other's work. Your manager needs to accept that if you are the sole QA checkpoint, things are going to get missed. Nobody is perfect.

In terms of confidence between you and management, you need to have an audit trail to back yourself up with. You should already be having management sign your test plan off before you start a project anyway, but how about getting a senior QA or analyst to look over it too?

Last thing I'd say is that if you've made it this far, you have what it takes to be a QA. Don't give up, you will get used to it!

Answer 10

Short summary: does your boss really expects 100.00000% working code as your output? Well, then he is incompetent, not you.

My honest advice would be: you cannot be perfect, so don't take it so seriously.

However, if you feel that the company is clearly trying to make you a scapegoat (so it seems), and that may hurt your professional career, resign as soon as you find equally or better paying job elsewhere. If anything, now is the time, when the economy is still strong.

I have a very strong feeling your boss is a terrible boss, especially for a software company.

In my field of work, I often come in contact with the issue of quality control. And more often than not, I hear bosses claiming that their workers are perfect (they would say, they just need to automate quality control because of some other funny reason, e.g. totaly unreasonable demands from their customers :) It seems that your boss is one of such people.

This is bollocks. People are not infallible, and certain factors make them even more prone to make errors. That's why QC exists! Your job is to catch mistakes done by other people, not to make things 100% OK, but to reduce occurrence of errors from say 99% to 99.9% If your boss does not understand then ... well, you will have many clashes with him at your current workplace.

If the errors you missed did not cause any problems in the time of the software run, then there is impossible to evaluate their impact, but it could be neglible.

If the company wants military-grade reliability, then they will need to change their process, which would probably mean employing other testers, and assign them redundant tasks (e.g. code testers cross-checking with others). Are they not willing to pay for it? Well, then they will not get 99.99%. Or 99.999% or whatever they aim for.

I think you are both the victim of the flawed production/testing process and the poor management process. But I don't think you are in position to change this.

Answer 11

Other answers have clearly presented the "there is simply no perfect software" fact of a software. My answer is based on your second point,

Should I have acted more clever and wise. Should I have not found bugs in Production and should have been quiet? Why did I highlight my mistakes​ and been noticed by management.

Yes, you should have acted more clever and wise.

What I like about you is the commitment beyond what is expected. I personally have faced such situations(other work area) to provide excellence but believe me its also about your true effort that only you know and others just take it granted as part of the job. I am not saying, it is not part of the job exactly but you have definitely done something extraordinary beyond what's expected.

What you should have done.

You should have instead let it slip by for maintaining management and stakeholders' confidence for the time being and corrected those in an official patch or next release. Because, hey! the code has been functioning normally for months.

However, the impact of those bugs must also be considered on the other hand. For ex, if your code operates a industrial machinery or an area involving public safety, then obviously the expectation is for a perfectly tested code. My answer assumes the nature of work to be not that critical. I am sure, you can decide on that yourself.

So, Don't quit.

You have a lesson in hand. Try to follow the best practices other answers have suggested and be cautious about ruining your image just because others don't understand.

Answer 12

Software gets released with bugs all the time. It's normal. Sometimes little bugs are in production for years before anyone even notices. If you found the bug before the customers noticed, then you've made a win for the company.

It would be great if I could write perfect bug free code, but I can't. That's why I need QA. It would be great if QA always found every bug, but you can't, because you're​ no more perfect than I am.

Speaking as a developer, I wouldn't want you to have a personal crisis, I'd simply want you to report the bugs to me so I have a chance to fix them.

Answer 13

No system built by human hand is perfect. Look at how many times your apps are updated on your phone - and these are all release candidates.

I have worked in a company that sounds similar to yours in that the senior management want a product that was 100% correct; delivered without any form of plan (i.e. 'I want this new feature now but you need to complete these other tasks as well'); and expected it all yesterday without additional incentives such as overtime etc. They failed to understand that these demands were not only impossible but also demoralizing.

They also criticized if we failed or bugs were found, so much so that we started to hide the fact that when we found bugs, we only corrected those that helped our working environment and left bugs that would have improved the client-facing parts of the product. Spiteful, childish and very bad practise.

They also didn't listen to advice on introducing anything as remarkable as Agile Development, Sprint Cycles or even some compromising form of project management and delivery cycle - despite the software teams best efforts to introduce and use such tools.

If this is familiar, you could do what I did - I left the company.

Answer 14

Making mistakes is a part of life. Accept that fact and don't be disheartened. Each time you go through this situation you are more careful next time. It horns your skills and adds to professionalism. Take it as a valuable lesson learnt.

Answer 15

It depends on the risk. What was the costs associate with these defects very high? Is the company now bankrupt? Did you kill a number of people? If not everything is fine. People make mistakes, it is only human.

Now the next step is not to quit, but to be responsible and learn from those mistakes. Do a root-cause analyses and prevent similar mistakes from happening again in the future. You are the best to prevent this as you have experienced and learned from these mistakes. Hiring someone new might just lead to same mistakes again as they go trough the same learning cycle.

You say you are a senior analyst, but probably you are just an average medior like most people. Just because your title says your an expert, doesn't mean you are one. Real senior experts and master have written books on the topic, not you, or did you? You and me and everyone else still learn on a daily basis. This is a good chance to learn, grab it and grow to the next level to become a real master.

Quality is a shared responsibility. You didn't make the mistakes. The developers did, or did they? It was the boss putting the team under pressure, or was he? Testing is just the last line of defence, expect mistakes to come through, unless you practise formal verification. Something no normal company can afford.

Answer 16

1. Accept the fact that bugs can occur in production.
2. Understand the bug and add it in your test case management and bug reporting tool.
3. Make sure that you do not miss it next time.
4. Understand from the bug that why it was missed and understand user's perspective towards the software from that bug.

Answer 17

1. Manure happens. Fix it and move on.
2. I wouldn't fire you. At least you care. A lot of DEVs and QAs do not.
3. You will find bugs in production and that is fine.
4. It is better when you find bugs. If the customer finds them, then be worried.
5. If a manager is giving you a hard time, they aren't looking at the big picture. There are already enough bad managers out there micro-managing everything.