How to respond to a suggestion from management to stop writing manual test cases

Question

Thought to ask this question while answering another question about automated test coverage.

Our management, is planning to get rid of manual test cases almost completely. And has started implementing below approach in some of the projects.

I will later explain what do I mean by almost.

Note: Most of the project in the organization are being implemented using Agile methodologies (mainly scrum).

Here is what the management suggested:

The development and QA team go through the requirements (in user stories) and discuss the wireframes/mock ups (if applicable). The development team will plan and finalize the object ids which they will share with the testing team (development team need to do it in first 1 or 2 days when they start coding).

Testing team will analyze the requirements and start writing automation scripts using the object ids shared by the development team.

Each team will have some manual testers too (I am not sure about the exact figure but around 20% of the total testing team)

So, as soon as a requirement is ready for QA, automation testers will check if all the identifiers are same as shared by development team. If not, then the tester will have to update the identifiers wherever required (because developers might have faced challenges while coding and have changed the identifiers in some cases).

With these minimal changes, automation test suite is ready for execution.

Now, there might be some scenarios that cannot be covered by automation testing. So, manual testers will write test cases to cover such scenarios during manual testing. And we also want to have testers who test the look and feel of the application, usability testing and do some ad-hoc testing. That's why I wrote almost earlier. Even during the time when automation testers are updating the scripts; manual testers can quickly do some high level testing during that period.

Now, I am in a dilemma regarding this approach. Because realistically, in agile projects, testers get very little time for testing (at least in some of the companies I know). Developers mostly end up taking more time than planned and testing window is squeezed. So, I am not sure if there will be enough time to update the automation scripts and do everything as per new approach.

So, just thought of discussing with comrades in this community to get their point of view on this approach. If this is good or not. Or if you can suggest any changes in order to improve or further streamline it.

Answer 1

Automated tests never will, and never should replace manual testers.

Automated tests are stupid. Even calling them "tests" makes no sense. They are automated checks which check for the conditions coder considered, and ignore everything else, however obvious for a human.

Compared to this, manual tester (because they are sapient humans), following a test script, or doing exploratory testing, will note unexpected conditions.

Pareto principle says that 80% of users depend on 20% of functionality. Automate that, and run those tests often. Rest (80% of possible paths through your system) is not used often enough to be worth automating. And automate checks for reported bugs, to check for regressions.

Also, you can automate only what you can do manually, and did manually few times. So by definition, new functionality has to be manually tested first, and only then can be automated.

Answer 2

I can understand your concern and in my view you can't just completely replace manual testing with automation. As is evident from your post that they are still happy to keep some of the manual testing so keep an eye on the things. For each sprint:

1. Perform a detailed analysis of the defects found by manual testing vs. automated testing along with their severity.
2. Keep record of the manual tests written/performed.
3. Record details of any Exploratory/ad-hoc Testing performed.
4. Issues found in the builds after the sprint completed
5. Issues missed by manual/automated testing

This should give you enough facts to present your case later on. As I learnt from Automated Web Testing By G.Suden, the goal of automation should be to increase the test coverage and not to replace manual testing! An automated test is as good as it is coded. So do your analysis after a few sprints and take it from there. Good Luck!

Answer 3

This is why using some sort of BDD tool makes a lot of sense, like Cucumber. You get 'executable specifications' that may or may not be automated, so you can combine information about requirements and test cases which everyone on the team should be able to read. It can be tough though.