Skip to main content
Software Quality Assurance & Testing

Return to Answer

added 6 characters in body
Source Link
Zach
Zach
  • 381
  • 2
  • 6

To answer your direct questions: You are correct that there is no universal process to identifying and/or fixing an issue's root cause. But this is obvious (if this task could be truly automated we wouldn't need developers, let alone QA!) so I think maybe you are looking for something like: what can be done to improve the quality and accuracy of root cause analysis? Or, how deep does a root cause analysis need to go to be valuable/sufficient?

Where I work, as part of our root cause analysis template we are required to answer "5 Whys". Or, what are the first five contributing factors to the issue? An example might be:

  • What? Customers cannot order our product.
  • Why(1)? Customers receive a 404 error when navigating to the eCommerce website.
  • Why(2)? DNS for the eCommerce website is not pointing to the expected IP
  • Why(3)? Our ISP has intentionally re-pointed DNS.
  • Why(4)? Our ISP has closed our account with them.
  • Why(5)? We haven't payed our ISP's bills in 6 months.

What I like about this system is that the specificity of your final "Why" is proportional to the complexity of the issue. The above issue is high level and conceptually simple, and so the "Whys" proceed quickly to an equally high level root cause. But when you perform this task on more code heavy issues you will find that the "Whys" quickly become much more low level, which matches how the issue will be addressed.

The "5 Whys" approach also gives the analyzer a clear stopping point (this is important because without a stopping point a QA resource doing a RCA is just a developer fixing a bug - which is inefficient for most teams) In the above example, we could certainly go to "Why(6)" and "Why(7)" if we wanted to (The bills are sitting unopened on Joe's desk -> We fired Joe 6 months ago). But itthis is probably not necessary. By the time we have reached "Why(5)" we already know what the root cause of the issue is and have a good idea who to escalate to. This works the other way too. Some complex issues will not be nearly as clear by "Why(5)". This is OK, because it serves as an indicator that the analyzer is either out of their depth, or the issue is broad enough in scope that an early hand-off is warranted.

To summarize: Consider a "5 Whys" approach to root cause analysis to constrain the scope of the analysis activity. In my experience this approach provides a deep and targeted analysis of issues which can be readily probed, while minimizing the time an analyzer can spend "spinning their wheels" on issues that turn out not to be easily pinned down.

Note: The "5" in "5 Whys" is not the magic. "5" works well for us, but it is easy to imagine that for other teams or processes "3 Whys" is a more sensible approach.

Note #2: If you do use this system, you must always provide all X "Whys". It is tempting to stop when "Why(4)" is as concrete as, "The for loop has an off-by-one error." But this is a mistake! If we're only a few levels deep in the "Why" stack this means that the causes leading up to the issue were not numerous enough to justify the issue occurring and their may be a process/architecture/whatever improvement available to us. Find the final "Why". For us it is often, "There was no unit test for XYZ."

To answer your direct questions: You are correct that there is no universal process to identifying and/or fixing an issue's root cause. But this is obvious (if this task could be truly automated we wouldn't need developers, let alone QA!) so I think maybe you are looking for something like: what can be done to improve the quality and accuracy of root cause analysis? Or, how deep does a root cause analysis need to go to be valuable/sufficient?

Where I work, as part of our root cause analysis template we are required to answer "5 Whys". Or, what are the first five contributing factors to the issue? An example might be:

  • What? Customers cannot order our product.
  • Why(1)? Customers receive a 404 error when navigating to the eCommerce website.
  • Why(2)? DNS for the eCommerce website is not pointing to the expected IP
  • Why(3)? Our ISP has intentionally re-pointed DNS.
  • Why(4)? Our ISP has closed our account with them.
  • Why(5)? We haven't payed our ISP's bills in 6 months.

What I like about this system is that the specificity of your final "Why" is proportional to the complexity of the issue. The above issue is high level and conceptually simple, and so the "Whys" proceed quickly to an equally high level root cause. But when you perform this task on more code heavy issues you will find that the "Whys" quickly become much more low level, which matches how the issue will be addressed.

The "5 Whys" approach also gives the analyzer a clear stopping point (this is important because without a stopping point a QA resource doing a RCA is just a developer fixing a bug - which is inefficient for most teams) In the above example, we could certainly go to "Why(6)" and "Why(7)" if we wanted to (The bills are sitting unopened on Joe's desk -> We fired Joe 6 months ago). But it is probably necessary. By the time we have reached "Why(5)" we already know what the root cause of the issue is and have a good idea who to escalate to. This works the other way too. Some complex issues will not be nearly as clear by "Why(5)". This is OK, because it serves as an indicator that the analyzer is either out of their depth, or the issue is broad enough in scope that an early hand-off is warranted.

To summarize: Consider a "5 Whys" approach to root cause analysis to constrain the scope of the analysis activity. In my experience this approach provides a deep and targeted analysis of issues which can be readily probed, while minimizing the time an analyzer can spend "spinning their wheels" on issues that turn out not to be easily pinned down.

Note: The "5" in "5 Whys" is not the magic. "5" works well for us, but it is easy to imagine that for other teams or processes "3 Whys" is a more sensible approach.

Note #2: If you do use this system, you must always provide all X "Whys". It is tempting to stop when "Why(4)" is as concrete as, "The for loop has an off-by-one error." But this is a mistake! If we're only a few levels deep in the "Why" stack this means that the causes leading up to the issue were not numerous enough to justify the issue occurring and their may be a process/architecture/whatever improvement available to us. Find the final "Why". For us it is often, "There was no unit test for XYZ."

To answer your direct questions: You are correct that there is no universal process to identifying and/or fixing an issue's root cause. But this is obvious (if this task could be truly automated we wouldn't need developers, let alone QA!) so I think maybe you are looking for something like: what can be done to improve the quality and accuracy of root cause analysis? Or, how deep does a root cause analysis need to go to be valuable/sufficient?

Where I work, as part of our root cause analysis template we are required to answer "5 Whys". Or, what are the first five contributing factors to the issue? An example might be:

  • What? Customers cannot order our product.
  • Why(1)? Customers receive a 404 error when navigating to the eCommerce website.
  • Why(2)? DNS for the eCommerce website is not pointing to the expected IP
  • Why(3)? Our ISP has intentionally re-pointed DNS.
  • Why(4)? Our ISP has closed our account with them.
  • Why(5)? We haven't payed our ISP's bills in 6 months.

What I like about this system is that the specificity of your final "Why" is proportional to the complexity of the issue. The above issue is high level and conceptually simple, and so the "Whys" proceed quickly to an equally high level root cause. But when you perform this task on more code heavy issues you will find that the "Whys" quickly become much more low level, which matches how the issue will be addressed.

The "5 Whys" approach also gives the analyzer a clear stopping point (this is important because without a stopping point a QA resource doing a RCA is just a developer fixing a bug - which is inefficient for most teams) In the above example, we could certainly go to "Why(6)" and "Why(7)" if we wanted to (The bills are sitting unopened on Joe's desk -> We fired Joe 6 months ago). But this is probably not necessary. By the time we have reached "Why(5)" we already know what the root cause of the issue is and have a good idea who to escalate to. This works the other way too. Some complex issues will not be nearly as clear by "Why(5)". This is OK, because it serves as an indicator that the analyzer is either out of their depth, or the issue is broad enough in scope that an early hand-off is warranted.

To summarize: Consider a "5 Whys" approach to root cause analysis to constrain the scope of the analysis activity. In my experience this approach provides a deep and targeted analysis of issues which can be readily probed, while minimizing the time an analyzer can spend "spinning their wheels" on issues that turn out not to be easily pinned down.

Note: The "5" in "5 Whys" is not the magic. "5" works well for us, but it is easy to imagine that for other teams or processes "3 Whys" is a more sensible approach.

Note #2: If you do use this system, you must always provide all X "Whys". It is tempting to stop when "Why(4)" is as concrete as, "The for loop has an off-by-one error." But this is a mistake! If we're only a few levels deep in the "Why" stack this means that the causes leading up to the issue were not numerous enough to justify the issue occurring and their may be a process/architecture/whatever improvement available to us. Find the final "Why". For us it is often, "There was no unit test for XYZ."

Source Link
Zach
Zach
  • 381
  • 2
  • 6

To answer your direct questions: You are correct that there is no universal process to identifying and/or fixing an issue's root cause. But this is obvious (if this task could be truly automated we wouldn't need developers, let alone QA!) so I think maybe you are looking for something like: what can be done to improve the quality and accuracy of root cause analysis? Or, how deep does a root cause analysis need to go to be valuable/sufficient?

Where I work, as part of our root cause analysis template we are required to answer "5 Whys". Or, what are the first five contributing factors to the issue? An example might be:

  • What? Customers cannot order our product.
  • Why(1)? Customers receive a 404 error when navigating to the eCommerce website.
  • Why(2)? DNS for the eCommerce website is not pointing to the expected IP
  • Why(3)? Our ISP has intentionally re-pointed DNS.
  • Why(4)? Our ISP has closed our account with them.
  • Why(5)? We haven't payed our ISP's bills in 6 months.

What I like about this system is that the specificity of your final "Why" is proportional to the complexity of the issue. The above issue is high level and conceptually simple, and so the "Whys" proceed quickly to an equally high level root cause. But when you perform this task on more code heavy issues you will find that the "Whys" quickly become much more low level, which matches how the issue will be addressed.

The "5 Whys" approach also gives the analyzer a clear stopping point (this is important because without a stopping point a QA resource doing a RCA is just a developer fixing a bug - which is inefficient for most teams) In the above example, we could certainly go to "Why(6)" and "Why(7)" if we wanted to (The bills are sitting unopened on Joe's desk -> We fired Joe 6 months ago). But it is probably necessary. By the time we have reached "Why(5)" we already know what the root cause of the issue is and have a good idea who to escalate to. This works the other way too. Some complex issues will not be nearly as clear by "Why(5)". This is OK, because it serves as an indicator that the analyzer is either out of their depth, or the issue is broad enough in scope that an early hand-off is warranted.

To summarize: Consider a "5 Whys" approach to root cause analysis to constrain the scope of the analysis activity. In my experience this approach provides a deep and targeted analysis of issues which can be readily probed, while minimizing the time an analyzer can spend "spinning their wheels" on issues that turn out not to be easily pinned down.

Note: The "5" in "5 Whys" is not the magic. "5" works well for us, but it is easy to imagine that for other teams or processes "3 Whys" is a more sensible approach.

Note #2: If you do use this system, you must always provide all X "Whys". It is tempting to stop when "Why(4)" is as concrete as, "The for loop has an off-by-one error." But this is a mistake! If we're only a few levels deep in the "Why" stack this means that the causes leading up to the issue were not numerous enough to justify the issue occurring and their may be a process/architecture/whatever improvement available to us. Find the final "Why". For us it is often, "There was no unit test for XYZ."