Do we need to consider all manual test cases for coming up with total Automatable & Non Automatable test cases

Question

I am leading the manual testing topics of our native mobile application. I have been tasked to give a report to management where we say total manual cases, total automatable cases and total automated.

This way the management can see the progress of automation.

Manual Testing Approach:

We have total of 618 Manual test cases as a singe suite & each case will have a Priority (1 or 2 or 3) associated with it. For Story tests, apart from the newly created story test cases, we also consider all P1+P2+P3 from the particular module which is related to/impacted by the story. End of the release, for regression we consider all P1 cases from all the modules. Only if it is a major release then during regression we consider all P1+P2+P3 from all the modules.

Automation Scope:

While looking for Automation, I feel that it is not worth to spend time on considering the P2 and P3 cases because of two reasons

1. Those will be executed once in blue moon [We do only minor releases 99%]
2. Most of them are actually throw away test cases which were created as part of the User Story testing.

What we want to achieve in our Automation?

We want to address the pain areas of QA where some P1 cases need to be tested against 20+ languages and kill the manual testing time.

Since we do not have different suites like Smoke, Regression, etc., we need to give our testers the test cases in the single suite only to identify what is automatable and not. Is it worthwhile to consider the lower priority test cases as well as the high priority test cases?

Answer 1

Start again by answering yourself and management (the answers may differ) why do you to automate anything. Remember that automation has its cost and many times it's higher than manual tests- developers time is precious, and you will never reach 100% automation (whatever it is) so some manual testing will be necessary anyway. You can easily end up with automation maintenance and bug fixes killing the automation developers time.

Where to start ? as someone smart once told me- start under the streetlamp, choose the easiest and fastest thing to automate and do it. It will bring the fastest ROI, and more important will build trust with the developers and management. While doing it you will develop the infrastructure, and if done correctly each new scenario will enable a few more by code re-usage.

Try to avoid complex and fragile scenarios, it will suck your resources both in development and maintenance and even worse will kill your management and developers confidence in your automation.

Having said all that, you have a good chance of finding out too late that your automation efforts are lagging behind, breaks often and that you need to chase developer and architects so automaton stays in synch with the actual product.

The solution might be what Alan Page calls Modern testing, his principles might be somewhat oversimplification of the problem and solution but the general idea is that YOU (your team) will help the developer write the tests themselves.

You might need someone to kick off a test framework and few example test cases, and help with maintenance and education for the long run. Obviously you will need your management support for "wasting" developers time especially at the beginning but for the long run this is probably the best sustainable solution- you wrote it, you test it.