Should most of the Regression testing items be automated?

Question

I work on a very complex financial product. We have very few Regression items/test cases automated. Like 20% is automated. Now, should I automate more and reach till 70-80 % coverage.

Is it necessary to automate most of the regression. It takes lot of time and effort to make a new project and debug it. Is the appropriate based on ROI and cost/benefit analysis? The months I will spend to automate regression suite can be used in testing stuff manually?

Answer 1

I can NOT answer in one word for this. Many other factors have to be considered to conclude anything :

1. How many tests from regression suite are repetitive?
2. What is the time horizon for this project?
3. Are you working on this financial product from Product development group or client group?
4. Are you going to deploy this project after few months?
5. How much bandwidth do you have?
6. Even though you could see RoI, you should check how much man hours are required and whether it's affordable considering resources, future plan and cost?
7. Do you have alignment on this with your superior or management?

Base on above points you can be in a position to take decision whether you should take step forward to automate possible tests from regression suite?

Answer 2

Two Thumb Rules

Rule No 1 - If you can automate, you should automate.

Rule No 2 - If you can't automate, learn it and follow Rule No 1.

See, before automation or even for testing anything manually, in both cases you require multiple things which are common like

1. Well-versed understanding of your application.
2. Your bandwidth to write the testcases based on your application requirements and assign them appropriate priorities based on the flows.
3. Execution of Testscases - for each release, may be feature wise or whole product/application wise which is kind of regression for the whole application or either you can run the feature wise testcases which will cover feature regression,right?

So if you see point 1 and 2 are the same for every project in each organization.

Now, Let's talk about the point number 3, The Execution of your tests

If you run anything manually, it is always going to take more time and the time of execution varies in proportion to the number of testcases and decreases with the number of resources who can execute these tests in parallel, right? - And This is already looking like a more time consuming process, right? and this effort you can multiply the number of times this activity you need to do.

Now let's talk about the execution with Automation.

You can't cover everything in automation, but you can cover most of the things in automation, which will make your life easy.

What/How to automate

1. Start with automating P0 tests by picking the most frequently used user flows from your application each week at least 2-3 or whatever number of testcases you can automate. Initially you may take time to automate but once you are in habit, most of the time adding new tests are just copy pasting and introducing the new params only, which will eventually take less time going forward. So now whatever pace of automation you want to maintain here, that will decide the total ETA of your automation completion.
2. Try to cover automation in phases not in single timelines. Decide what to automate in phase1 and then in phase 2 etc.
3. Keep on going and continue picking the priority P1, P2, P3 tests and so on.
4. Whatever you have automated, that will be reduced from your manual testing so you may be having more bandwidth for automation.
5. Last but not least - Automation is not a destination, it is a journey till your application is there. So don't worry about covering 70-80% but think about that you just need to keep going with automation.