Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

in the context of a good practice in context on how to perform a no blame post mortem review, with a view to process improvement.

If a major issue is found in an application after it has been released (a test escape) who/what should ultimately be considered responsible?

  • The developer for introducing the issue?
  • The tester for not finding the issue?
  • Both the developer and the tester?
  • No-one (it's the system that failed, not the people)
  • Everyone

Thoughts ?

Note: No actual persons were harmed in the writing of this quesiton

share|improve this question

9 Answers 9

up vote 28 down vote accepted

Short answer is "everyone" - as was pointed out in other responses.

For the long answer, I'll refer to an article I wrote recently on this exact subject, The One That Got Away.

share|improve this answer
    
+1 for the excellent article –  TristaanOgre May 31 '11 at 14:48
    
A very good article. "Getting hung up on playing the blame game when something unexpected happens is a waste of time." Nice! –  Joe Strazzere May 31 '11 at 16:22
    
A Very good article :). –  Aneef Apr 25 '13 at 16:24

I'm not sure what you mean by "responsible" in this context?

  • who should get yelled at
  • who should be fired
  • who should get demerits on their next annual review
  • who should take action to cure the problem
  • something else

Clearly both the developer(s) and tester(s), and almost certainly others, should be concerned whenever a defect escapes into production.

The tester should try to determine why it wasn't caught in test (assuming that it actually wasn't detected, rather than having been detected but having the bug deferred), and how to prevent such an escape from happening again. The tester also should be involved in reproducing the problem in the test environment, and verifying an eventual fix.

But clearly, the tester didn't create the bug, so the developer needs to understand the root cause of the bug, how to cure the bug, and how to prevent it from happening again.

Management (both Test and Project Management) might need to understand what led up to this defect escape. Perhaps aggressive schedules, inadequate/unclear Requirements, lack of training, etc, might or might not indicate that some processes need to be revised.

share|improve this answer
4  
+1 for including aggressive schedules and inadequate requirements. –  TristaanOgre May 31 '11 at 12:54
2  
+1 Code doesn't start with the coder & end at test. There are many cooks in the kitchen so to speak. Incorrectly understood customer needs, poor requirements, lack of equipment/tools, unidentified hardware, etc. can all contribute to defects. Use the post-mort to determine what slipped through & why so you can catch it next time around. –  CKlein Jun 2 '11 at 13:20

The whole team regardless of responsibilities should shoulder the blame. It is not a "test escape" but a "team escape". Testing is not only a department, it is also a discipline that everyone in the team should be involved in.

share|improve this answer
1  
Upvoting. It is at least fruitless and possibly destructive to assign blame to an individual. Moreoever, no non-trivial software is perfect. Fix the bug, and then if you want to assign blame, take a look at your processes and ask whether they need to change. –  user246 May 31 '11 at 19:32
    
+1 for emphasis on collective responsibility and teamwork –  Aruna May 31 '11 at 20:15
    
+1 Very true, this is the way I have always looked at it –  MichaelF Jun 1 '11 at 12:09

As said, everyone. This is why postmortems are important and valuable, as well as high-communication environments where people feel comfortable taking responsibility for problems without being afraid of blame.

Get everyone who might have had a chance to eliminate this issue - management, PMs, developers, testers, business owners, and so forth into a room. Brainstorm, come up with an action plan. Put someone (probably the PM or management) in charge of implementing that plan, and schedule a meeting to review progress after some period of time. The result is that people get credit for taking responsibility, rather than blame for being responsible for problems.

If you get blamed for being responsible, your goal will be to not be responsible for anything. A vacuum of responsibility doesn't produce quality.

share|improve this answer
4  
+1 "If you get blamed for being responsible, your goal will be to not be responsible for anything. A vacuum of responsibility doesn't produce quality." Love this! –  TristaanOgre May 31 '11 at 20:00

Where I work, which has a very enlightened approach to QA/Dev, we actually have a report that details every bug found at customer sites, why it was missed in QA and what steps are being taken to ensure that similar bugs will not be missed.

We understand that everyone is human. If we expect developers to occasionally wirte a bug into the code, we should expect QA to occasionally miss a bug too.

The trick is to have a process that catches these things as best we can. This is why we report coverage for tests, and test plans are checked periodically.

Blame is not a useful idea to work with. What we try to do is find all the failures that occurred in the process that caused the bug to escape. It usually takes 3-4 failures in the same path for this to happen. Then each failure is dealt with using the Kai-Zen method of 5-whys, if somewhat haphazardly applied.

Obviously, if one of the failures is a test that was reported as done but wasn't really done, we might need to give the person involved a stern talking to. The same goes for changes made in code without notifying QA that they should re-test.

As ever, this varies with the type of software, company size and culture and a thousand other things, so your mileage may vary.

share|improve this answer
    
+1 for this comment, but you should add that if there's a test escape, Bruce should be fired. –  user246 May 31 '11 at 19:38
    
@user246 LOL ... I'm lucky that this is just a hypothetical, discussion provoking question then :-) –  Bruce McLeod Jun 2 '11 at 3:12

All of the above...

The developer is responsible for writing buggy code

The tester is responsible for not finding the issue

Depending on who asks the question, and the purpose and meaning of "responsible" you can get a better answer using root cause analysis of the test-escape.

The tester missed the bug either by not testing it (coverage not good enough) or by accidentally missing it (human or automation error), this can be divided more to the existence and following of test development procedures (were there clear requirements ? who reviewed the documents ?) , test execution procedures or even work overload that can lead to errors.

If you wish you can continue digging even more (following the 5 whys rule), lets stay with a coverage issue for example- we want to understand who's responsible for why wasn't this case covered. Did anybody reviewed your tests and test implementation ? Are there clear procedures that every test document should be reviewed ?

Etc...Etc...

I think that few more questions will usually lead to one, and only one responsibility- god (didn't give the company's CEO enough brains to hire a good test manager).

share|improve this answer
    
I think when you note "coverage not good enough" as a rationale you also need to note schedules, sometimes the date is fixed and no manner of testing is going to get all your tests in a short window. You can try, but sometimes you have to pick and choose while dates slip. Sadly, this is still a case to consider. –  MichaelF Jun 2 '11 at 11:41

I hate workplaces where the reaction to a failure is to find a responsible. This does not encourage people to provide the best they can.

In my point of view, the correct action to take is to fix the problem: first the bug, then the process having led to the bug.

share|improve this answer

This is a great question.

Everyone is responsible. Dev - Test - Support - UAT. Instead of identifying who has missed, We need to look at

  • Why it is missed
  • What should we learn from it
  • Is it Repetitive / Permanent fix in terms of process change / Checklist
  • For the Developer - Identify Design Review Process / Unit Test cases which were missing and led to this issue
  • For QA - Knowledge Gap / Missing Test Cases / Root Cause of missing the issue

I would suggest use of orthogonal defect classification to identify the phase bug was introduced (design/ coding/ code review changes/ data issue / integration test issue /boundary cases). Next step is to fix the process / checkpoint for that phase/ identify lessons moving forward.

Advantage here is actual phase where bug was injected is also identified plus additional checkpoints can be added to identify/uncover it before code is deployed in production (UAT/Test)

http://www.research.ibm.com/softeng/ODC/DETODC.HTM

http://www.chillarege.com/articles/odc-concept

share|improve this answer

Though Architecture-developer-test-Tech Support teams are responsible for the Leaked defect, most of the companies have a ingrained culture of blaming the testers.

I had been in situation where the APIs were continusouly changed on a regular basis and there was no coordinated communication between the teams involved. Testers were continously changing the API test framework, which resulted missing out some of the scenarios which resulted in leak defects in the System Integration testing.

RCA was done badly, we just found the gaps in our test design & how to improve the test design, but never focussed on why we missed them during our test design

After taking the entire blame, we started setting up rules for our test team.

1) Feature Interaction matrix for all the APIs with respect to their interactions

2) Never change the Test design unless there is a published(approved by Dev. manager/Architect in consultation with Test ) API/Feature change and change should come through a Single Point of Contact.

3) Test Reviews to be done diligently and test team should discuss more with in ourselves as well as Dev. team on the reason behind the changes and what type of problems can arise due to the changes.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.