We are in the process of developing an in-house testing solution that is tailored to our applications. As of right now when you add a new test case to a plan you must choose the prerequisite steps, and then enter the steps, along with the additional details. My question is when creating a new test set should we ask if the cases within the set are meant to be stand-alone (requiring a log in, and log out after each case so the next case can execute), or just assume that all tests sets are developed to be run sequentially with no prerequisites necessary?
- Anybody can ask a question
- Anybody can answer
- The best answers are voted up and rise to the top
Instead of an "either/or", it is better to think of test case or scenario testing as a tree structure:
Trunk: This is common test set that are always run (e.g. launch the application)
Branch: These are major scenarios that have few common tasks (e.g. "admin" vs "user" scenarios)
Branch: ... (if needed)
Branch: ... (if needed)
Twig: Test cases that are launched from a common branch (e.g. "admin" sets up a new user with specific permissions.)
Leaf: Specific test cases to verify the permissions in a specific "twig" scenario (e.g. verify positive and negative cases for each permission granted or denied)
This allows you to use applications like VM or automated setups to snapshot branches or higher levels to repeat scenarios in sequence with variations and also the greatest independence of setups such that they can be run independently in any order.That structure gives you the maximum implementation flexibility for efficiently testing your application context.
Here are pros and cons of running tests in a sequence. I understood you ask in a context of automation, not manual execution.
This is a pretty common situation - and one I've dealt with myself. Based on your comments and the question itself, I can offer a few suggestions.
An additional theoretical concept to consider is this. The ISTQB terminology separates test cases from test procedures. During test case design, in the description of the test case you should detail all that you mentioned, prerequisites, etc and the expected outcome. However, when implementing the test case (either as manual or automatic) you essentially create the test procedure which may include details on how to execute the test case and also various optimizations such as combining test case parts if they are similar. So, strictly speaking your test case description should be stand-alone and you would create a separate test procedure detailing sequential execution. In practice, of course, you don't always document both, however this is a good practice to seprate more abstract concepts of a test case and its implementation details.