Should QA or tester mention the root cause for a bug?

Question

QA closes bugs where I work. In most cases, bugs are closed without mentioning any details about the root cause. The only info given is one of the options in our bug tracker, such as "code", "test data", "tester error" etc.

Senior QAs believe that details about root cause should be mentioned by the developers who fix the bug, not QA. I disagree because I believe that QA finding out root cause (even at high level) has the advantages mentioned below. Note that my "Contrived examples" are quite similar to issues which I have seen.

1 - It enables QA to learn more about the system or even business domain.

Contrived example: A search functionality in a web app was not working for one test data. For another test data, it was working. So, was the error in the search engine or something else ? From dev, I learned that indexing would simply fix the error. In the process, i learned about the system and how to trigger indexing.

2 - It improves QA's knowledge & understanding on how bugs can creep in when they/dev can miss some test cases. They can use this knowledge to come up with more robust test cases for other situations.

Contrived example: On a web page, after clicking one button, an api is called. In rare cases, the response from the api gets delayed and the side effect is that other UI elements get disabled. Those UI elements are not related to that api call and should not be disabled if the response is delayed. The dev told me that delayed response caused the issue. Next time, when I do UI testing, I will see if I should simulate delayed api responses or not.

I am not sure if these reasons are compelling enough to make it mandatory for QA to mention root cause details in bugs.

So, should a QA mention details for the root cause for a bug or not ? Please mention the reasons also.

Answer 1

It depends on the type of testing. For white box testing, maybe. For black box testing, no. The whole point of black box testing is that you're testing based on the specifications, not the code. A black box tester shouldn't care why the specifications aren't being met, just that they aren't being met. And knowing details of the internals may actually detract from black box testing.

What's more interesting for a black box tester is to know what sort of testing you were doing to drive out the problem. Load/stress? Recovery? Installation? Upgrading? That can tell you what sort of activities you may want to focus on in the future.

However, I'd caution you on a couple of things.

First, you need to come up with consistent, well-defined, and comprehensive list of root causes; if one developer puts in 'concurrency', another puts in 'threading,' and another puts in 'race condition', you aren't gathering useful data.

Second, you need to make sure that the root cause is just that, the root cause. In the cases you gave, I'd argue that what you listed for the root cause isn't the root cause. That a database that needed to be indexed wasn't the root cause; the root cause may be why said database wasn't indexed. Or it might be something beyond that. Similarly, that the API timed out isn't the root cause of the bug; you need to look at the why the API timed out.

And third, you need to make sure the data's being used, as Michael said.

You seem to be conflating two different things here: root cause analysis and learning more about the code under test. You shouldn't need to have an excuse to learn about the code under test, and you shouldn't do root cause analysis unless you're going to use the information for something.

Answer 2

The testing manifesto states preventing bugs over finding bugs. With this in mind I think you should do a root-cause analysis for each defect that was found. Afterwards find a strategy or experiment to prevent a similar issue in the future.

A root-cause is deeper than bad testing or sloppy coding. What is the real cause behind this? Lack of skills, time pressure or personal problems? Ask why at least five-times.

In your examples you still focus on the symptoms.

• The indexing is not automated, but why isn't it? Checking your indexes manual next time is just solving the symptom. I think you are missing a reproducible build, but still why? Lack of knowledge? Why?
• The responses are delayed, but why? Is extra testing solving the real cause here? Or can we prevent it earlier in the development-cycle?

So to answer your actual question. Yes, QA should also be involved in getting to the root-cause. No, the QA should not define the root-cause alone. Yes, I like to keep an extra field in our bug-tracker to contain the results of the root-cause analysis.

I think critical thinking QA people could be of real added value during the analysis. Whenever a defect happens I like to grab a developer, QA and business stakeholder and draw a root-cause mindmap and analyse the issue deeper. Do it together, share knowledge, solve real problems on the correct level. Timebox it, most of the RCA's are just a couple of minutes, unless it are company critical issues.

Answer 3

When talking about black box testing, I certainly believe that the root cause should be entered by the development team. Black box testers have good understanding of the requirements and the overall functionality to be tested. But, since they are viewing the system as a black box; just relying on the requirements. Because by definition, black box testing doesn't require the testers to have in depth understanding of the implementation from coding stand point. So, generally, project teams assign testers who are very thorough with the requirements, smart while thinking about the possible test scenarios but these candidates are generally less technical (if we compare with white box testers who dive deep into the code).Having said that it is very difficult for black box testers to identify the root cause of a defect when it falls under one of the following:

1. Flaw in design

2. Issues in Technical Approach

3. Insufficient knowledge

4. Oversight

5. Deployment Issue

Moreover, defect root cause is a very important field while doing root cause analysis and can be very helpful in identifying the areas of improvement for development team. So, it is better that it is reported correctly than being reported incorrectly for defects; as it will show misleading results when RCA is done for defects. So, in order to avoid this, it is better if the defect cause is entered by developers. Let me know in case you have any follow up questions.

Answer 4

Yes, when definitely known, have the QA enter it, but if not known, let the developer enter it once they figure it out.

However it really depends on a lot of factors:

• Can the tester really tell what the root cause is ?
• Does the tester have access to required resources such as source code ?
• Does the tester have the technical experience to diagnose the issue ?
• Is the bug about code or about configuration, data, dependencies, etc. ?

For items such as 'tester error' or 'test data' and the tester knows that, then it is great to capture that knowledge at that time by the QA recording that fact. If only a developer will know later then have the developer enter it when they figure it out.

Try to encourage this as part of a team approach. Whowever find out enters the info.

Review past bugs and categorizations to learn from them is a great idea. You can come up with new and useful to tests based on the knowledge. I would just separate that from the issue of who enters detail for a given ticket.

Also, reflect on what you do with that data and how it adds value to the organization. Communicate that information to the team so they know why it is important to capture it. If you can't clearly define the reason you capture it and the direct value it adds you have a different issue. Also for personnel performance review this may become a bullet point to assess.

Answer 5

I think its perfectly fine for a QE to investigate the root cause, for the learning purposes you mention. Learning is good.

However, I see RCA as a team sport - not exclusively QE responsibility. For issues that leak into production, the team made the mistake that caused the bug, and the team missed finding/fixing before release. So, the team should answer the 2 questions: 1) how was the bug introduced and 2) how did it escape?