Who is ultimately responsible for issues that are found after software is released?

Question

In the context of a good practice in context on how to perform a no blame post morten review, with a view to process improvement.

If a major issue is found in an application after it has been released (a test escape) who/what should ultimately be considered responsible?

• The developer for introducing the issue?
• The tester for not finding the issue?
• Both the developer and the tester?
• No-one (it's the system that failed, not the people)
• Everyone

Thoughts ?

Note: No actual persons were harmed in the writing of this question.

Answer 1

Short answer is "everyone" - as was pointed out in other responses.

For the long answer, I'll refer to an article I wrote recently on this exact subject, The One That Got Away.

Answer 2

I'm not sure what you mean by "responsible" in this context?

• who should get yelled at
• who should be fired
• who should get demerits on their next annual review
• who should take action to cure the problem
• something else

Clearly both the developer(s) and tester(s), and almost certainly others, should be concerned whenever a defect escapes into production.

The tester should try to determine why it wasn't caught in test (assuming that it actually wasn't detected, rather than having been detected but having the bug deferred), and how to prevent such an escape from happening again. The tester also should be involved in reproducing the problem in the test environment, and verifying an eventual fix.

But clearly, the tester didn't create the bug, so the developer needs to understand the root cause of the bug, how to cure the bug, and how to prevent it from happening again.

Management (both Test and Project Management) might need to understand what led up to this defect escape. Perhaps aggressive schedules, inadequate/unclear Requirements, lack of training, etc, might or might not indicate that some processes need to be revised.

Answer 3

The whole team regardless of responsibilities should shoulder the blame. It is not a "test escape" but a "team escape". Testing is not only a department, it is also a discipline that everyone in the team should be involved in.

Answer 4

As said, everyone. This is why postmortems are important and valuable, as well as high-communication environments where people feel comfortable taking responsibility for problems without being afraid of blame.

Get everyone who might have had a chance to eliminate this issue - management, PMs, developers, testers, business owners, and so forth into a room. Brainstorm, come up with an action plan. Put someone (probably the PM or management) in charge of implementing that plan, and schedule a meeting to review progress after some period of time. The result is that people get credit for taking responsibility, rather than blame for being responsible for problems.

If you get blamed for being responsible, your goal will be to not be responsible for anything. A vacuum of responsibility doesn't produce quality.

Answer 5

Where I work, which has a very enlightened approach to QA/Dev, we actually have a report that details every bug found at customer sites, why it was missed in QA and what steps are being taken to ensure that similar bugs will not be missed.

We understand that everyone is human. If we expect developers to occasionally wirte a bug into the code, we should expect QA to occasionally miss a bug too.

The trick is to have a process that catches these things as best we can. This is why we report coverage for tests, and test plans are checked periodically.

Blame is not a useful idea to work with. What we try to do is find all the failures that occurred in the process that caused the bug to escape. It usually takes 3-4 failures in the same path for this to happen. Then each failure is dealt with using the Kai-Zen method of 5-whys, if somewhat haphazardly applied.

Obviously, if one of the failures is a test that was reported as done but wasn't really done, we might need to give the person involved a stern talking to. The same goes for changes made in code without notifying QA that they should re-test.

As ever, this varies with the type of software, company size and culture and a thousand other things, so your mileage may vary.

Answer 6

All of the above...

The developer is responsible for writing buggy code

The tester is responsible for not finding the issue

Depending on who asks the question, and the purpose and meaning of "responsible" you can get a better answer using root cause analysis of the test-escape.

The tester missed the bug either by not testing it (coverage not good enough) or by accidentally missing it (human or automation error), this can be divided more to the existence and following of test development procedures (were there clear requirements ? who reviewed the documents ?) , test execution procedures or even work overload that can lead to errors.

If you wish you can continue digging even more (following the 5 whys rule), lets stay with a coverage issue for example- we want to understand who's responsible for why wasn't this case covered. Did anybody reviewed your tests and test implementation ? Are there clear procedures that every test document should be reviewed ?

Etc...Etc...

I think that few more questions will usually lead to one, and only one responsibility- god (didn't give the company's CEO enough brains to hire a good test manager).

Answer 7

I hate workplaces where the reaction to a failure is to find a responsible. This does not encourage people to provide the best they can.

In my point of view, the correct action to take is to fix the problem: first the bug, then the process having led to the bug.

Answer 8

This is a great question.

Everyone is responsible. Dev - Test - Support - UAT. Instead of identifying who has missed, We need to look at

• Why it is missed
• What should we learn from it
• Is it Repetitive / Permanent fix in terms of process change / Checklist
• For the Developer - Identify Design Review Process / Unit Test cases which were missing and led to this issue
• For QA - Knowledge Gap / Missing Test Cases / Root Cause of missing the issue

I would suggest use of orthogonal defect classification to identify the phase bug was introduced (design/ coding/ code review changes/ data issue / integration test issue /boundary cases). Next step is to fix the process / checkpoint for that phase/ identify lessons moving forward.

Advantage here is actual phase where bug was injected is also identified plus additional checkpoints can be added to identify/uncover it before code is deployed in production (UAT/Test)

http://www.research.ibm.com/softeng/ODC/DETODC.HTM

http://www.chillarege.com/articles/odc-concept

Answer 9

I find posting the blame is the wrong way to think about it but to answer your question it could be a single person or everyone. I.e could of been a business spec that was not communicated to the dev team or it could be the tester that did not test a specific COS appropriately or everyone for that matter.

Whats most important to me as a QA lead is what can we do to prevent it from happening again. After every roll out we review leaked bugs and analyze what we could have done to prevent this. Our company currently separates the bugs into the following category.

• Requirements(COS)
• Unit Testing(Low level dev testing)
• Acceptance testing(Verification of COS)
• Regression testing(Post QA related functionality)
• Automated testing
• Other
• Not sure

I find currently a lot of our recent bugs have fallen into the regression section. Using this model has really helped our team analyze our strengths and weaknesses in terms of preventing bugs being released.

Answer 10

Though Architecture-developer-test-Tech Support teams are responsible for the Leaked defect, most of the companies have a ingrained culture of blaming the testers.

I had been in situation where the APIs were continusouly changed on a regular basis and there was no coordinated communication between the teams involved. Testers were continously changing the API test framework, which resulted missing out some of the scenarios which resulted in leak defects in the System Integration testing.

RCA was done badly, we just found the gaps in our test design & how to improve the test design, but never focussed on why we missed them during our test design

After taking the entire blame, we started setting up rules for our test team.

1. Feature Interaction matrix for all the APIs with respect to their interactions

2. Never change the Test design unless there is a published(approved by Dev. manager/Architect in consultation with Test ) API/Feature change and change should come through a Single Point of Contact.

3. Test Reviews to be done diligently and test team should discuss more with in ourselves as well as Dev. team on the reason behind the changes and what type of problems can arise due to the changes.

Answer 11

Outside-the-box answer: Wrong question.

Feces occurs. We don't celebrate it, but it happens. Your development team's response needs to be driven not by some abstract sense of responsibility, but by its concrete policy about fugitive bugs.

Your team should have such a policy. It could be as simple as "ask the boss to make a decision" or a pre-ordained flow-chart. Each person has a role. Do your role and stop asking about responsibility or blame.

Root-cause analysis is a legitimate activity in the aftermath. This is a very different matter from assigning categorical responsibility for leaks to a specific subgroup. Sure, mistakes were made and natural persons made them. Maybe the natural persons in question had a bad day or need training (or sleep). At any rate, keep the personal details between the QA team and the project leadership, for rumor mills are far more troublesome than just about everything else combined.

But if you must have an answer: the process did it. The process can be improved. We are always following the process but it might not match whatever's written in some document somewhere.