Automate the stable build only

Question

I have the understanding that Automation Scripts need to be built on a stable build (application). I mean once manual QA signs off, to reduce their burden on regression/repeated testing, we Automate test cases.

It is not fair to ask the Automation team to build scripts on developing the build.

However, I don't have a best practices article or a page from the Automation Community to support my statement. How can I convince a demanding client?

It looks like a workplace question/situation but it isn't. It is actually (and literally what I am looking for) about Automation best practice (automate stable application) and any supporting document/page in its favor.

Answer 1

I don't know if an authoritative answer can be provided, but I think simple logic should suffice:

• your primary use of automation is regression
• developing automation scripts takes (a lot) more time than doing the same thing manually

If the above is true than

• it makes sense to invest (a lot of) time in automation on something that will last enough time to break even (timewise)
• it doesn't make sense to invest (a lot of) time in automation of something that is temporary (unstable application version)

Answer 2

It is actually about Automation best practice

There are good practices in context, but there are no best practices.

I have the understanding that Automation Scripts need to be built on a stable build (application)

Why? What is a "stable build" in your context?

Automation in Testing serves to enhance human capabilities in testing. These capabilities will depend on the people and their goals.

Anyone saying "This is the Way" is inherently ignoring context and inevitably will suggest inefficient approaches.

The best practice for you will be whatever will make your team to reach its goals faster, more cheaply, and with higher quality (value to your clients).

If your team is suggesting to have new automated checks in "development builds", ask "why?". Maybe the cycle time to have a new "stable build" is too long and people need the information in sooner. In this case, should the team decrease the cycle time or create automated checks for "development builds"? Only you folks can answer through the process of Plan-Do-Check-Act.

Answer 3

The basic goal for automation in your case seems to be take the load off QA. However, as pointed out the automation scripts have to be written again over a short period of time (unstable build). This is basically not effective, and will cost more in terms of investment. In my opinion, a better way to convince the client would be to stress on the point of bad investment if tests are automated at this stage as compared to when they will be automated once a more stable build is available. You can try to present some estimate statistics. The following qa testing companies links might be helpful,

https://blog.qasource.com/resources/-6-test-automation-best-practices-and-tips-for-startups

https://blog.qasource.com/4-essential-api-testing-automation-best-practices

Answer 4

If you need to provide any kind of reference, I'd go with the syllabus for ISTQB Test Automation exam.

https://www.istqb.org/downloads/category/48-advanced-level-test-automation-engineer-documents.html

See chapter 1.1 Purpose of test automation

Objectives of test automation include: Reducing the total test cost

Automating a not stable app build can require additional effort/rework/maintanence, therefore from an economic perspective, this approach will be more expensive.

From chapter 6.1 Criteria for Automation

Suitability of automation for the stage of the software product lifecycle. An SUT has a product lifecycle which can span from years to decades. As the development of a system begins, the system changes and expands to address defects and add refinements to meet end user needs. In the early stages of a system’s development, change may be too rapid to implement an automated testing solution. As screen layouts and controls are optimized and enhanced, creating automation in a dynamically changing environment may require continuous re-work, which is not efficient or effective. This would be similar to trying to change a tire on a moving car; it’s better to wait for the car to stop. For large systems in a sequential development environment, when a system has stabilized and includes a core of functionality, this then becomes the best time to begin the implementation of automated tests.