How to determine applications that are valid candidates for UI Regression/Automation Testing?

Question

I have been coding my scripts in the past year and demo'd my automation scripts to him, executed tests in front of him and explained that I will be adding more features. Then he asked me, what is the purpose of my automation tests? I said for regression testing, since I am working with different apps at the same time in a week, why not give this a push? It seemed that he wasn't convinced with it and thought I can just manually test everything - I don't like manually testing everything and this is the reason why I wanted to jump into the Automation task.

I continued coding for a year, determined test cases that are repetitive. I finally demo'd my final product to some of my teammates and received positive remarks - but deep inside, I feel this is not enough. My boss has not seen the final product yet, and the concept of my selenium tests did not change since I have created the framework. Now his questions bugs me - do I really need to automate my tests? What are the scenarios that I could consider to get back and maybe convince myself that I need the automation tests.

I am the only QA in 10 developer team and some of them are not really happy having a QA because they think QA is redundant since they do their Unit and Integration tests..

Answer 1

I determine applications that are valid candidates for UI Regression/Automation Testing by using the following factors:

• Revenue and profit
• Amount of functionality
• Expected product lifetime
• Business domain complexity
• Length of time to manually test
• Unit and integration test coverage
• Ratio of manual testers to the devs
• Complexity of the functionality to test
• History of missing bugs by manual testing
• Experience and availability of manual testers
• Cost of manual testers compared to automation
• Number of devices, browsers and versions to test on
• Workflow and whether manual testing holds things up

Answer 2

Every application which is executed repeatedly is a candidate for Automation.

As any engineering task, you need to balance pro and cons: automating a task means writing code which needs to be maintained as the underlying application changes.

Obligatory XKCD cartoon

If application is short-lived, or is undergoing series of rapid changes, automating it might consume more resources than it saved, so fully automating it does not make sense.

Even in such situations (when fully automated regression tests are not appropriate), I found that certain steps still can be automated, to benefit the manual testers.

If you apply pareto principle, you get 80% of the benefit of the automation from automating 20% of your tasks. So start there, and try to automate more as you have time (which you usually will not).

Answer 3

Every application is a valid candidate for Automation. At certain points along the development phase, applications become bloated with functionality. Certain areas of said applications are easy candidates to be chosen to automate as they become more prone to regressing into various unwanted behaviors and having a framework in place to quick check those parts is best practice.

Now to tackle your question...

1. Do you really need to automate your tests? No, not really.

2. Would it be better for you to do so? Absolutely YES! Why? It becomes a huge time-saver for you and your team, the amount of time spent otherwise checking huge chunks of functionality are easily monitored now by checking the outcome of your Automated tests.

And judging by the setup of the team you are currently assigned to, I`d assume that you need all the free time you can get. The workload, coming off of 10 developers to one tester, can't be little.