I have been writing Test Cases for half a year as of now. My question is: How many test steps are allowed only in one test case? For example more than 6 steps is not allowed or depends on functionality to test.
-
7Test cases are a documentation - a way of communicating an idea. Textual limits in communication only occur in specific situations, such as the physical limit of a newspaper or the arbirtrary number of words in an essay. In software testing there are no such constraints. The important aspect of any documentation is to communicate its message well. Particuraly for the realm of testing, I would suggest looking into Cem Kaner's work on Bug Advocacy.– João FariasAug 26, 2020 at 14:42
-
+1 @JoãoFarias Also, the number of steps does indeed matter... but it is a side effect of how you are organizing your cases. Focus on quality test cases and that will keep step numbers low. For example you'll constantly be refactoring yor code to avoid the build up of such anti-patterns as too many steps for a case– Michael DurrantAug 27, 2020 at 10:50
-
Measure the effect but change the source is one expression I've been playing with.– Michael DurrantAug 27, 2020 at 10:51
6 Answers
4500.
That's my estimate. Some say 6750, others 500, others yet wouldn't go over one million steps per case.
Jokes aside, there is no fixed number nor there is someone who would set the limit. It is pretty vague anyway what is a single step case.
For example, you could write a test case in this way:
- Login to the app
- Add 2 items to cart
- Confirm 2 items are in the cart
or, this way:
- Turn on the computer
- Open browser
- Navigate to app url
- Click username field
- Enter username
- Click password field
- Enter password
- Click login button
- Add 2 items to cart
- Confirm 2 items are in the cart
They are the same, but the first one is much more legible (and BDD styled). Also, notice that each step could be thought of as a single test (1. test if the computer can be turned on, 2. test if the browser can be opened, 3. test that navigation is working ...)
-
9
Using the word allowed suggests there is or should be a fixed rule. I think what you are looking for might be a guideline.
The guidline would depend on factors like the context, the domain complexity, the risks, the level of skill of the people that execute the test-cases, etc..
I would suggest that you start with a max of 10 steps. Once you go over it you can challenge yourself, maybe the test-case:
- is to detailed or verbose
- is testing multiple functionalities and can be split up
- is so long because the testability of the application is poor
- is so long that it makes it hard to understand what it is really checking
- etc...
Now decide on a case-by-case basis if it contains to many steps or not. After a while re-evaluate your guideline and decided on a new threshold for challenging yourself.
No. of test steps does not matter but few things matter which can make test cases highly effective:
Single Test Goal: Every test should be focused on testing a single requirement only which can be directly mapped to the acceptance criteria of the story.
Clarity: Every step in a test case should be clear enough so that anybody in the team can follow it.
No Duplicate steps: There should not be any kind of redundancy between test cases in terms of steps or validations.
Sub- test cases : If there are some common steps/navigations then that should be abstracted out in separate sub test cases to be called out(hyperlinked) in main test cases.
Single Pass/ Fail reason: Every test should pass/ or even fail for a single reason which is the primary objective of the test case.
Assumptions, Preconditions and Post-Conditions: Test cases should include all assumptions that apply to a test, along with any preconditions that must be met before the test can be executed.
Provide test data: Identifying and preparing test data can sometimes take the most time in testing. When possible, provide the test data to be used for the test case within the test case description or with the specific test case step.
Traceability: When possible, link the test cases to appropriate user story or requirement by unique ids.
Tag functional area: Have keywords to attach to your test cases related to functional modules so that it is easier to search related test cases.
Automation mapping: If have automation in place related to the manual test case link it with unique ids for future Traceability.
I personally came across many lengthy test cases written by others that try to cover so many things and can potentially fail for multiple reasons which are confusing.
In a nutshell, tests should be small & to the point, focused on testing a single requirement.
-
The "reason" a test case fails might be interpreted as a bug in the code, and you can't write your tests such that every possible unique bug has a matching test case. Therefore, every failing test could have many reasons in the code. I suspect that you mean that each test case should check one property of the outcome. E.g.
output size is less than 4 kB
.– MSaltersAug 28, 2020 at 9:48 -
In my view, a test should be highly focused to test only one thing. If a test fails for various reasons , I would suspect its too broad in scope(test design issue) and more likely would be "overlapping" with other tests as well. Sep 3, 2020 at 0:29
Test steps are written to log the paths to test or reproduce the bug. There is no such recommendation that test steps can't be more than 6. No of test steps varies based on test cases and test scenario. If the scenario is complex and tough to understand, then so many test steps will be required. Test steps should be simple and specific so that another tester/developer can test the test case. It shouldn't be confined in numbers.
Steps are written for both tester and developer both. It helps the developer to reproduce the issue and helps the tester to re-test the issue.
Write clear and simple test steps, it'll reduce time to find out, solve and re-test the issue in the quickest way.
Test cases are a documentation - a way of communicating an idea. Textual limits in communication only occur in specific situations, such as the physical limit of a newspaper or the arbirtrary number of words in an essay.
In software testing there are no such constraints. The important aspect of any documentation is to communicate its message well. Particuraly for the realm of testing, I would suggest looking into Cem Kaner's work on Bug Advocacy.
There is no hard and fast rule for assigning steps for a particular test case. I believe it depends on the product functionality as well as the engineer's approach to QA validation.
To quote an example: We want to test checkout feature on a website
One approach is to:
Create a single test case for verifying 'Purchase of a product'from the website and,
that test case would end up resulting in around 10-15 steps
However, the other approach is to segregate the whole purchase flow by splitting the test steps into multiple test cases (Each Testcase which will now contain less number of steps):
Testcase1: Verify Login into application
Testcase2: Verify category selection and search for the product you want to buy
Testcase3: Verify product view and browse the details
Testcase4: Verify product quantity, color, or any other attributes that need to be selected
Testcase5: Verify 'Add to cart' operation
Testcase6: Verify 'User details' operation
Testcase7: Verify 'Checkout' operation
Testcase8: Verify Payment process
Testcase9: Verify the success page once the order placed successfully
Testcase10: Verify Email/Messages confirmation sent to the customer
Some will prefer the 1st approach as the whole flow can be tested under a single test-case whereas some might prefer the latter one as it provides a better understanding of the passing/failing areas within the flow.
To elaborate further, let say our validation fails at a step where a user unable to perform only 'Add to cart' operation for a 'Z' Category product but working fine for other categories.
- With the former approach: A single test case is marked failed with a particular category comment and hence, gives an assumption that the whole checkout flow is failing.
- However, with the latter approach, only Testcase5 will be marked as failed, and a QA engineer can easily identify and track which sub-areas are working fine or not in the checkout flow.
These days most of the software testing services company opts the latter approach for manual testing services.
Apart from the test case steps, a quality test case should also define a few other parameters like the Preconditions, Priority, Expected Results, Product Area, Type: Smoke/Sanity/Regresison, etc in the details for better understanding of the scenarios.