Test strategy to bring automated tests from UI to lower(API/Unit) layers?

Question

Background/Problem Statement: Recently I came across a project team, which is having thousands of UI automated tests covering all kind of manual tests including positive, negative etc., and obviously struggling to maintain them with every release with very high execution time.Basically ice-cream cone anti-pattern. There is a rest API based backend to support multiple application platforms.

Possible Solution: I am thinking to correct the Test Pyramid by bringing a substantial number of tests on lower layers and reduce maintenance & execution time.

Help required: In similar scenarios, what is the strategy my fellow QA experts follow without impacting agile tempo.

Answer 1

A key strategy for me is to convince the business of what needs to be tested where, otherwise... they'll end up directing testing everything through the UI... So the two key main points I am making to my business are the need for test automation that performs well in terms of two key factors:

• Speed

• Reliability

Success in both speed and reliability for software test automation is hard and most technology companies struggle with both.

Speed:
The overall goal of software test automation is to provide faster feedback to the business at all levels from the development team to the various levels of management, appropriately summarized at each level.
For software test automation itself, using the UI to create test data or set application state is incredibly slow and in some cases not even possible and will thus not provide the timely feedback that is required. This is frequently a 10x+ (and often 100x+) factor in terms of testing speed and dramatically affects the ability to both develop and test automation. If automation takes 20 minutes to reach a specific spot in a workflow instead of 7 seconds this will make quality automation development impractical. Consider also then trying to run that same test in 20 different device-browsers when that means 20x7 = 2+ hours for that 7 second test. Imagine wanting to try 10 different changes to get it right (common).
Reliability:
We do not control or select our customers device, browser, version or other client-side characteristics such as different javascript and rendering engines. Customers devices are now updated daily in the background, communication is asynchronous and network connectivity may vary. This greatly limits the ability to have reliable automated UI tests and ultimately prevent us from having 100% reliability despite out best efforts. This is not a problem that any one company created and it is not a problem that one company can solve - through we can allow and account for it. One of the main implications is to have as few UI tests as possible in favor of Unit and Integrated tests that run in seconds instead of hours and days (test pyramid vs ice-cream cone). The few automated UI tests we have should be carefully selected to ensure they avoid duplication and test data combinations through the front end.

Internally, for test automation development, it is essential to observe and structure testing based on the test pyramid

and the agile testing quadrants

to ensure getting the benefits of swift feedback throughout the the development process.

There are two key technical problems to solve:

• setting up test conditions and state without using the UI
• controlling test and application state through APIs

Traditionally the need for test automation has not be enough of a critical business success factor to justify actually spending serious application development effort, time and money on addressing these two key technical issues (they are certainly solvable with enough effort and determination). That has now changed and test automation is increasingly recognized at a hugely critical business success factor underpinning so many other efforts and plans of today's businesses.

For the detailed strategy of dividing up unit, integrated tests and UI tests, a few thoughts come to mind?

• Create some examples of functionality with (e.g.) 60 unit, 12 integrated and 4 UI tests to show the test pyramid in action
• Make sure unit tests mock and stub dependencies
• Ensure that application and automation engineers share code reviews of each others work
• Expose application environment stability for functional tests constantly in the application/automation team space area
• Expose test status information constantly in the application and automation development area
• Ensure that application developers write units tests and use code coverage and quality measurement tools as a required part of their continuous integration code development process.
• Ensure that application developers and UI automation developers are physically close to each other for easy drop-in chats.
• Consider creating a formal triage system (e.g. during backlog refinement) to determine what to test where (in terms of unit, int and uat tests) at least until you can do it instinctively.

Answer 2

1. Estimate working load, how many test cases are there to be migrated from UI tests?
2. Estimate how much time you can spare in each sprint and discuss with your team leader and your fellow members regarding you want to spare some time in each sprint to work on test case migration.
3. Prioritize automated UI tests, there is a chance a portion of them are no longer needed. Migrate highest-prioritized tests first.
4. Based on your discussion from step 2, allocate work load to yourself during each sprint.
5. Keep old UI tests running even after they have been replaced, phasing them out slowly. You may not be the only person who is running them, keeping every stakeholder updated.
6. Document your approach as there are thousands of UI tests, it is unlikely someone will come in to help you or you will leave before you finish; good documentation will help others continue working on test case migration.

Answer 3

If I would approach to the goal like yours I would:

• Introduce some proxy between your front-end and back-end that would log the requests to your Back-End
• Run all your tests for UI
• Parse the proxy logs so that I have all the endpoint calls grouped by the endpoint addresses (and possible the parameters - depends on what cost of the migration we can afford)
• Having all the things grouped I would spend significantly less time to implement the API tests. Probably I would even introduce the converter that would build the tests automatically basing on the grouping obtained on the previous step.

Answer 4

Adding to @Michael Durrant's very good answer.

If you are looking for a quick win, I have come up with an idea for converting selenium tests to API tests. This will only work for cases in which you have "UI" tests that have nothing to do with UI logic and are only used to test backend logic.

The idea is to capture the network traffic when running a selenium test in a HAR file, and covert this HAR file to a replayable API test - which involves identifying the request-response correlations and adding assertions.