3

This is how our test case development process looks like. A tester writes a test case in a spreadsheet, with each test step and expected result. Then, another tester reviews the test case and approves it if its ok. The test writer then executes the test when needed.

The problem with the above approach is that sometimes, the test reviewers approve tests even when the test steps are missing some crucial information. This omission was discovered only when a third person was asked to execute some tests.

So, I was thinking of changing the process by making the reviewer execute the test. I believe that this might make the reviewer be more thorough because he has to execute the test, i.e. he cannot afford to miss important details. As an aside, the new way might bring out lack of documentation. When, reviewers find that system documentation is needed in order to run a test, then they will ask for it and mention it to our QA manager (during scrum meetings).

Does the new approach make sense ? If not, then are there any other approaches that we could try ?

3
  • To me your approach doesn't make much sense, to be honest. If you are trying scrum - you mention this - than I wouldn't expect this. I feel there is a lot of 'overhead' in your approach, apart from being tedious. And that time is not being used to test. Why is the 'tester' not executing test him/herself to begin with? How do 'testers' know what they check is actually what is needed? How does the team know? Does automation play a role in your situation?
    – Ray Oei
    Oct 14, 2017 at 0:08
  • @RayOei - Why do you think there is a lot of overhead and its tedious ? Currently, testers execute the test themselves, after another tester reviews the test. Why do you ask about automation ? We automate some of these (initially) manual tests if needed.
    – mrjobs
    Oct 16, 2017 at 18:14
  • I would say that some of the answers given here, might provide food for thought ;-) In an agile environment I would expect a lot of interaction, communication and sharing of information between all involved (coders, testers, business alike) so they understand & share context, assumptions and expectations as much as possible. In an agile environment I would expect a lot of automation to cover basics and repetitive work so testers can focus on the hard parts...
    – Ray Oei
    Oct 16, 2017 at 19:42

4 Answers 4

2

Making reviewer to execute the test could catch some omission but it would not be 100%, ever.

Manual test are executed not by computers, but sapient, thinking humans. Humans can think, learn, see patterns. Computers, not so much (they can, but it requires deep learning).

Difference is the unsaid assumptions, based on the experience of the human (which is different for every person). Something obvious to one person is far from obvious for another.

Even if I write directions how to execute steps for myself, when I try to execute them a year later, I often find I forgot to mention some "obvious" steps. So I add them to my instructions, and year later, sometimes it is again not obvious even to me, and I might need more investigating.

And opposite is also true: sometimes too much details might be annoying. Like in those "for dummies" books, when they repeatedly explain some trivial details again and again, even if I learned it 20 pages ago, as if I was really a dummy who cannot remember a thing.

Best test cases has right amount details for the audience who will be using the script but not more.

2
  • Yes, I feel that my approach cannot catch 100% omissions. Actually, it might not even catch much, let alone 100%, especially if writer and reviewer think alike and leave a lot of unsaid assumptions that might be needed by them one year later or even by a less experienced tester. But, I hope that with my approach, reviewers will be a little more careful, instead of just signing off on test reviews to get it off their to do list.
    – mrjobs
    Oct 16, 2017 at 18:17
  • "hope springs eternal in the human breast". Good luck. Oct 17, 2017 at 13:56
1

This reminds me one of Agile Manifesto:

Individuals and interactions over processes and tools

If the missing one are "crucial information" or "important details", seems your approach will be a patch at the result but not at the root cause. When the important details are missing from testing ( or even in developing phase ), this is an issue that need to be addressed with team:

  • Do staffs really understand story, its benefits and impacts to the system?
  • Do they realize this information from the first place?
  • Are these information/detail clear communicated during the requirement phase and documented in a story management tool you use?
  • etc - any other question to identify the issue root cause.

If an answer is no, so you know the area that need to be fixed rather than the additional-cost-review-process. You can get feedback from team and keep improving it.

If answers to these questions is yes, they do understand the story. and the detail is clear communicated and documented. I believe this kind of issue should be small.

However I still see benefits of reviewing for test approach, test technique, test coverage with corner cases or functional/non-functional/security test to gain more confidence in product not the important detail of a story.

Hope it helps,

0

Does the new approach make sense?

Maybe, but how do you know for sure? Measure the number of testcase that where incomplete for a while. Change the process and measure again. Is it better now? This might be a bit time consuming, but your team is probably already doing something similar for defect leakage.

Better is to hold a systems thinking session. Thinking about how your current systems influence the number of incomplete test-cases, finding loops that have a positive effect and try to make those better. Define some actions and experiment. Monitor your process/system and keep improving it. This means adding and removing steps in your process/system.

Be careful of adding more gates to your process. Do you really want to make your process more heavy and time consuming? The goal is not better test-cases, but less defects in production, maybe there are totally different means to reach your goal.

My advice: Experiment, evaluate, adapt continuously. Start trying. Use a PDCA cycle.

3
  • The systems thinking article seems complex to me. Instead, I wished they would give a simple explanation of what is systems thinking, an example of why its needed and then explain the modelling, flow diagrams, feedback loops, graphs etc. (PS - To someone who does not know what a car is, you don't start by telling them its a complex system consisting of electrical, mechanical components etc. You can start by showing them an example of a car, say that it lets you go from place A to B. Then, you dive into combustion/electric engine, mechanics etc. Makes sense ?)
    – mrjobs
    Oct 16, 2017 at 18:50
  • You are right, the LeSS example might be a bit overwhelming, but it covers most of what you need to know to get started. Watch this video for a more simpler example: drawtoast.com (A simple and fun introduction to Systems Thinking) Oct 17, 2017 at 8:32
  • Do keep in mind software development is most of the time a complex system according to the Cynefin framework. en.wikipedia.org/wiki/Cynefin_framework Oct 17, 2017 at 8:33
0

Moving one work-product from one person to another one just for validation is not what you can call "agile". If you wish to get the maximum value for your product, then align your process toward this goal.

So instead of having one person write the test cases and another one validate them, make them work together to define what needs to be tested and how. This way you'll move most of the issues detected during the validation/execution to this single phase which will eliminate all the back and forth and the related documentation.

Then have one of them write the tests cases matching these tests requirements and have him run them at least once. There's no real value in validating the test cases since they can be adjusted during the execution and mainly because you'll get a feedback if something is missing. So I would skip the validation if the way the tests are executed has no impact on people's safety. Instead, monitor the issues and give the necessary training if the level becomes unacceptable.

Bottom line is you'll get more value by having people working together toward a common goal compared to simply ask them to execute a single task with a vertical mindset.

3
  • I am trying to imagine how two testers could work together instead of one writing and other one validating. Are you suggesting that they meet/chat and come up with testing scenarios ? If yes, then do you think it will be better if they come up with the scenarios before they meet/chat ? After this, they can check if there are any gaps in coverage and figure out the steps for testing the scenarios.
    – mrjobs
    Oct 16, 2017 at 18:40
  • Also, why do you think our current system is not agile ? How do you think it will become more agile by using the approach you suggested ?
    – mrjobs
    Oct 16, 2017 at 18:53
  • 1
    The whole point of scrum is to have a fast feedback. Having people work together on the same problem at the same time is more productive than having them work separately at a different time. I'm not saying that they should write the tests together. What I'm suggesting is to have them work together on the scope which justify the validation step in your current process. It's hard to say without knowing exactly what is the purpose and added value of this validation step.
    – Florent B.
    Oct 16, 2017 at 20:03

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.