6

Should Testers/QA do Root-Cause analysis of every Defect/Bug they report to Developers? Well, I perform black-box testing on AUT. Sometimes I try to drill-down the issue when I find bugs. Should testers find the reason for bugs by analyzing the source code? Or should testers report Defects/Bugs/Problem immediately without any analysis?

It takes lot of time and effort to do it. I mean, is it effective and productive for testers to find real cause of all bugs or leave it to development team to fix them and when it's fixed...then understand it properly?

  • 1
    How far does root-cause go for you? A real root-cause could be a developer who had a daily hangover, because he has financial problems due to a gambling addiction... – Niels van Reijmersdal Feb 3 '15 at 13:11
  • We've all experienced that root cause too many times. – Paul Muir Feb 3 '15 at 16:47
  • All you should be doing is understanding the problem "well-enough" to describe the problem and hopefully to be able to tell how to reproduce the problem. More often than not, anything beyond that is just duplicate work that the developer is going to end up doing anyways and they likely would have nailed down the cause much quicker than you since they would hopefully be more knowledgeable about the code. With that said, it can be very helpful to have someone from QA who understands the software somewhat, plus it probably makes the job a little more interesting, so in moderation is fine. – Dunk Feb 3 '15 at 23:09
8

It's not necessary for QA to do root-cause analysis for every defect they found. It's up to circumstances. And for the level of drill-down, I did not drill to the level of source-code when I tested a Unix binary application. But I did html/css root-cause analysis when I tested a website. As a view of user I represent, I access only the information that a user can access.

It's good for a QA to do root-cause analysis - I back it. More information provided helps developer to fix it faster. Tester has more understanding about the system under test. Tester could find the critical area that never tested or discover additional test scope during analysis. And tester who do analysis might have the solution for that defect!

But in another point of view, yes, it takes a lot of time and might affect the overall test project. Tester might take time to find nothing helpful or someone in the team knows the root-cause already. So it's important to encourage tester to keep the balance.

If I take care of a testing project, I will encourage tester/QA to consider the following circumstances:

  1. Defect Description
  2. Test Coverage & Test Progress against Test Plan
  3. Defect Priority & Defect Severity
  4. Team Availability

How?

  • Defect Description First of all, when tester found a defect, then should take time to have enough information; environment, replication steps to allow them to recreate the defect.
  • Test Coverage & Test Progress against Test Plan If the high priority test cases have been tested and the progress is good against test plan, tester could do further analysis for a defect. But if there are high priority or critical test cases outstanding, tester should take time to continue test that area more than root-cause analysis a defect.
  • Defect Priority & Defect Severity If a defect have high severity such as system crash with simple steps or high priority defect such as it affects more of the user. tester should take time to analysis to make it fix as soon as possible.
  • Team Availability If developer is busy with another issue and tester is available, it's good to take this time to investigate the issue before submit all information you get to the developer.

The last important I will mention here is communication. When you work in a team, before you take ( or will take ) a lot of time on any issue - communicate to team to see if they have any concern. You will have a good balance.

Hope it helps, :)

  • "discover...area never tested...additional scope" That's possibly the single biggest reason to investigate, especially on a legacy application -- there's no telling what secret bugs have been written off "working as designed" for years because no one took the time to investigate. – EKW Feb 4 '15 at 0:09
7

As others have said, it depends.

My usual method works like this:

I find something that doesn't seem right:

  • If it's really obvious (a typo in a high-profile part of a site, an error message that gives the line of code that's failed), I'll check with a developer to see if they're working on that code right now (this is often the case in agile or other rapid development methodologies) or have a fix checked in that will be available in the next build.
  • If there isn't anyone working on the obvious problem, I create a quick defect report and it goes into the system.
  • If the problem isn't so obvious, I'll do some research to see if I can reproduce the problem, if this is a long-standing issue or not, if someone else has reported this or not, and so forth.
    • If there's a pre-existing defect report, I'll add my information to it.
    • If I need to create a new report, I'll do so, with reproduction steps and my best guess as to the approximate area of code that's likely the source of the problem.
  • If the developers have problems reproducing, I'll do more investigation, and at this point may end up digging into the code (I mostly work with a web application driven by classic ASP and using a lot of stored procedures) to find the region of code that's likely the problem. I'll use tools like the browser developer tools to find problematic JavaScript or hidden server errors, and SQL profiling to get a list of queries to step through manually and trace the most complex stored procedures.

I've got a fair amount of coding experience and started out as a developer - which means while I'm not in practice enough for application coding, I'm capable of tracing through code and working out where problems lie. Not every tester can - or is willing to - do this.

My advice is to do what works for you and your team. Testing is something that has no one "right" answer, and no simple solution. If it works well enough for the team and the customers, it's good enough.

  • I like the way you established the steps to your process. I find that different teams work differently but in an idea/perfect scenario this should be the generic format for such issues in my opinion. – Paul Muir Feb 3 '15 at 16:48
1

Nobody can really answer it since it depends.

Depends on your qualifications, your and your team work load, the dev team work load, the complexity of the issue etc.

I work now in a combined engineering type team and I still can't give a definite answer to your question, your best bet is to discuss this with your colleagues and managers.

1

To echo others, it depends and I'd say that it largely depends on the culture at your company. It ultimately comes down to a trade-off between developer time and tester time.

The more time you spend digging into the root cause of an issue, the less time you'll spend looking for other issues. However, you will save the developer some time because he/she will have more information to work with.

The point is that someone will be spending time on the deeper analysis. If you find the right balance that works for your team, that's all that really matters. As a tester you could go a step further and actually fix the problem too. . . which is how you become a developer, which isn't a bad thing if that's your goal. But someone still needs to do the testing!

0

I won't take anything away from what has already been stated, but I noticed that your question implies that your company may still be in the "start up" phase. If your Marketing and Executive offices intend to go "IPO" ("Initial Public Offering" meaning: you'll be listed on the stock market), then you may have some work to do. Before that happens, you must have a well defined Policy, Process and Procedure(s) in place. (Policy is the company's mission statement. Like; "We combine Hard and Soft engineering to produce intelligent widgets". Process is the interface between different departments or work groups. A procedure defines a single work element. Like: "root cause analysis".) Before you go IPO you will have to have all three PPP's in place and DOCUMENTED. This is so that you can get ISO 9000 certified. (Desirable for Marketing.) Basically this means that you do what you say and you say (in the documentation) what you do. An ISO 9000 auditor will look for deviations between what the documents say you do and through observing what you REALLY do. If there is a difference, you will register a deviation and will have to reschedule with ISO to come back after you have fixed the deviation(s). This "could" upset the higher-ups if they are on a tight timeline to go public.

Long story short, if you have a Quality Manager in your company you need to ask her for any process definitions that apply to the interface between your workgroup and design.

  • Is this answer for this question? I do not see the word IPO in the question, maybe you misplaced this answer? – Niels van Reijmersdal Feb 3 '15 at 13:38
  • I'm having a real hard time with this answer. Firstly, I don't see why this doesn't apply to "established" companies (perhaps more so since startups tend to be lacking in the QA department.) Secondly, I'm not sure "ask your manager" is a good answer - for all we know the OP is the Quality Manager and is looking for guidance to help set the PPPs for the firm. Perhaps you could better align your answer to the question at hand: if you were said QM how would you approach this problem? – corsiKa Feb 3 '15 at 17:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.