Should you create a manual test script before automating it?

Question

Is it best to first create a manual test script for a newly developed feature and then automate it? and then link them up for tracking purposes?

Or it's not necessary to create an automated test without creating a manual test script?

One problem I see with the first approach is that you may not always be able to automate the exact manual test script therefore you might have to create a manual test script based on the automated one(the other way around). To that end, I think you could just decide which part of the feature is ideal for automation and which is a good fit for manual testing?

Any advice would be greatly appreciated!

Answer 1

There's no one answer to this. It's highly dependent on the context in which you are working.

For example, I've worked in fields with varying degrees of regulation, where traceability from requirement and change through test case execution is vital. In these cases, having a test case documented in a test case management tool and selecting appropriate test cases for automation tends to be a good solution. Having the test case in the test case management tool also has some advantages for keeping track of the test results as the tools I've used integrate to allow for tracking when the test cases are executed, what they are executed against, the pass/fail status, and attach output (such as screenshots, console output, or log files).

There are several factors to consider, though.

Some tests are "one-off" tests. They may need to be executed very infrequently. If they aren't part of the standard testing suite, they can have a lower priority for automation. However, something else to consider would be the difficulty in setting up and executing the tests - cumbersome tests, even those that are infrequent, may benefit from automation.

There are also different levels of tests. Unit tests, integration tests, and acceptance tests don't all need to be treated the same. The more formality tends to be around system tests and acceptance tests. Still, as your system's criticality or the risks surrounding failures increase, you may want more stringent (and perhaps even bi-directional) traceability around requirements/changes, tests, and code.

You should look at your context. Do you get any benefits by writing tests case up for manual execution before automating? Do those benefits outweigh any costs associated with adding those steps? What about the risks or costs of not writing the tests for manual execution first? Does creating the written test script before automation imply that the test must be run manually, or is it just a way to link a test script with an outside tool? If your system scales, does the process scale (with both increased system complexity and any organizational changes needed to support the more complex system)? Many questions to consider.

Answer 2

In testign everything is context depended . Its hard to suggest merits and demerits unless you states the following things like :

1. What methodology you work in (Agile or waterfall ?)
2. What is your sprint length
3. How much time you gets for testing
4. What is your responsibility ( Is it only manual , is it automation or is it both)
5. What is the Team size
6. What is the release plan

But i would recommend following things as best practice

1. Adapt a TDD where you create test cases parallel to the development. So when the development sprint starts , start your test sprint in parallel and use the time to create the tabular testcases
2. Don't wait till you get the actual product to write the testcase and start testing it
3. Demand more time for documenting test case in tabular format if current sprint size is insufficient
4. First start with just writing down all the use cases that comes to your mind ( means just the title ) . Else you will feel the work tedious . First write down all use cases and once you are satisfied with the use cases you have decided, start writing the test steps and other details in tabular format
5. Split features between team members , give each team member the responsibility of testing a particular feature. This will give you more time to focus and document test scenarios for that specific feature than getting nervous about entire product.
6. Keep initial few days for test documentation , and start execution only after it is completely done. Don't do it in parallel else you lose focus while testing

Demerits:

1. If you are a single QA in a small sprint size team , this will be really hectic tasks and you might lose focus on actual testing and affect test quality
2. If you are single QA responsible for both automation and manual , then this will be a really waste of time and effort . It should be a complete no for this approach . Just copy paste the test script title if anyone really want to know what use cases are being tested . If they still insist demand more members into team

Answer 3

No

As everyone says, in reality "it varies".
So let this be the "no" answer for better or worse and for all to vote on.

My take is no: Requiring a manual script first, before automating is wasteful and adds delays. Some of the scripts will be thrown away. Other automation tests are hard to describe manually. Be lean in all things and only create stuff you plan to use on an on-going basis when possible.

I also say no because, from your description, you are talking about creating and permanently storing the manual scripts and linking them to automated ones. This will quickly lead to a maintenance nightmare both within each manual script and also in the manual-automated synchronization.

I recommend relying mostly on automated tests as living documentation.
For manual tests themselves, checklist are definitely great.

Answer 4

There is no specific rule for it, as the eventual decision (manual test first or automation script) boils down to the situation. As a guideline for software testing services, one can say that for a newly feature, the manual tests comes first, even before the development happens. This is in accordance with TDD. These tests usually act as metric for the developer and the tester on how well the feature is implemented. Later, based on these manual tests, the automated script can be written. The only question at this stage being whether it is optimal to automate or not. However, in practice this order is mostly not followed as situation usually demands flexibility in the work approach.

Answer 5

What usually works best for me, mainly in small test teams with tight schedules, is developing test automation and manual testing together. You start by testing exploratory in a not-very-scripted way as a way to learn the product, as you go you start taking notes and build a more scripted approach (or at least more organized). At the same time you start slowly automating small parts of the scripts, have the automation only login with all the IDs and URLs hard coded, add verification to the page title, move constants to a separate file, start writing using a POM etc. You start by using those bits of automation as force multiplier and as tools, but slowly as you gain confidence in the test code you will start using it continuously and as part of an automation system.

There are some underlying assumptions hidden here though of which the most important is that you should have your test infrastructure and environment already ready and familiar.

Answer 6

It is not necessary to create a manual script but you definetly neeed to make some test design exersize first. 1: Create a checklist to identify what is planned to be tested/automated. I would suggest to use some automation-friendly naming convention for such a checklist.

2: Analyze the checklist and identify where the detailed scenario is needed.

3: Write a detailed test scenario using some automation-friendly DSL (Gherkin is a great example).

Note: In the same time, I would not advise using BDD tools for testing: they exist for other purposes.

Answer 7

It is usually a lot of effort and hard to impossible to derive business logic from tests alone. That's why you need a point of truth outside of your test. This can be anything from requirements of the whole application to a test description for a singular test - even a backup of the test at the time of creation can help. The closer this point of truth is to your testing purpose, the shorter the way to answer doubts about the validity or purpose of the test.