1

Background: Combining separate Test Procedure and Test Case documentation has advantages, but so does leaving them separate. IEEE std. 829 mentions system level of complexity as a contributing factor to the decision to combine. From experience, it seems the tradeoff in terms of complexity is whether including test procedures makes the suite too cumbersome to navigate. Reused test procedures that are lengthy may slow down the tester when they could instead repeat a procedure from memory. On the other hand, switching between documents can be slow.

Has anyone reached a more concrete point of view on the tradeoffs of these approaches and a method for deciding between them?

0

I have wrestled with this problem a number of times.

For instance, I have had a number of occasions where I needed to hand each member of a group of testers a stack of paper to describe the steps they had to take.

  1. The first time a tester needs a detailed procedure it has to be there for them to read immediately. Similarly, when the procedures are very similar but differ in important details, the procedure needs to be right there.
  2. Subsequent times, I don't fold the procedures into the test script, but simply refer to it by a distinguished name.

I have written a number of different tools to handle this problem, each with its own set of quirks and annoyances, for me and the people doing the testing.

My current vision of a test documentation package involves generating fairly sophisticated HTML. For instance, it might use disclosure triangles or pop-ups or pop-overs to allow the tester to view the details as often as they need in order to become familiar. The basic test documentation looks short and straightforward, with most of the detail hidden until needed.

In a similar vein, the web page with the test procedure in it would be an active form where items could be checked off as they are completed, and details written down when needed, and anomalies or bugs recorded at the exact point where they were noticed in the script. This requires some non-trivial back end software. I have looked into using a survey generator/capture tool for this kind of support of manual testing.

  • #2 makes sense. Having to track down the procedure may be a reason to put it in a test procedure or test plan that could be referenced. Your solutions cover the concerns, it's just the mental debate that makes me want to step back to make sure I'm not overlooking something that could be more efficient. Software enhancement is another solution as you noted. – user176692 Oct 23 '14 at 20:06
  • Keeping SUT complexity in mind, the std. also notes a single TP could be shared among mult. TCs in a combined doc. Marking as answer for #1 and #2 addressing doc separation concerns. Will separate TP solely based on complexity, will include once for section if shared and simple, or for a specific TC if simple. – user176692 Oct 23 '14 at 20:36
1

Since you have said it yourself that having separate documents may add to tester's time, wouldn't it make sense to have a checklist instead of such vast and mostly not so easy to read documents. The checklist can be used for testing and whatever information is gathered during testing can be properly formatted in a document so that the intended stakeholder can easily read and understand it!

  • Checklists show that some of these documents may be outdated or inefficient. They are very effective, though I'm not sure they're detailed enough on their own. They may require other documentation or verbal instruction about the SUT or what is expected. With extra info, they would start to look like a test suite probably. This extra info would even be helpful for troubleshooting automated test suites. Will never stop using checklists though, agreed. – user176692 Oct 23 '14 at 20:14

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.