0

I have the understanding that Automation Scripts need to be built on a stable build (application). I mean once manual QA signs off, to reduce their burden on regression/repeated testing, we Automate test cases.

It is not fair to ask the Automation team to build scripts on developing the build.

However, I don't have a best practices article or a page from the Automation Community to support my statement. How can I convince a demanding client?

It looks like a workplace question/situation but it isn't. It is actually (and literally what I am looking for) about Automation best practice (automate stable application) and any supporting document/page in its favor.

2

I don't know if an authoritative answer can be provided, but I think simple logic should suffice:

  • your primary use of automation is regression
  • developing automation scripts takes (a lot) more time than doing the same thing manually

If the above is true than

  • it makes sense to invest (a lot of) time in automation on something that will last enough time to break even (timewise)
  • it doesn't make sense to invest (a lot of) time in automation of something that is temporary (unstable application version)
2

It is actually about Automation best practice

There are good practices in context, but there are no best practices.

I have the understanding that Automation Scripts need to be built on a stable build (application)

Why? What is a "stable build" in your context?

Automation in Testing serves to enhance human capabilities in testing. These capabilities will depend on the people and their goals.

Anyone saying "This is the Way" is inherently ignoring context and inevitably will suggest inefficient approaches.

The best practice for you will be whatever will make your team to reach its goals faster, more cheaply, and with higher quality (value to your clients).

If your team is suggesting to have new automated checks in "development builds", ask "why?". Maybe the cycle time to have a new "stable build" is too long and people need the information in sooner. In this case, should the team decrease the cycle time or create automated checks for "development builds"? Only you folks can answer through the process of Plan-Do-Check-Act.

7
  • Stable build means QA signed off, no more enhancements, all manual tests passed, all bugs resolved.
    – paul
    Apr 14 at 8:43
  • @paul: that sounds like the QA is a gatekeeper, last step in the process. and that sounds like a blaming culture when a bug pops up in production. "all manual tests"... hm, you perhaps mean "all tests we imagined at the time of creating them"? exhaustive testing is not possible, hence "we executed all tests" doesn't ever exist. "all bugs resolved"... I'd be skeptical about this as well, perhaps again "all bugs we found have been resolved".
    – pavelsaman
    Apr 14 at 12:40
  • I follow @pavelsaman's points - such wording may be very dangereous. Additionally, I still think a reflection on the question I left by the end of my answer, inside your context, rather than looking for an authorative, context-ignorant, suggestion. Apr 14 at 12:45
  • 1
    I don't see any skeption (what's the right word :) ??) here. Let me take an example: Login functionality - 1.Test cases were written as per Acceptance Criteria, 2.All are passing, 3.all bugs that were logged during are closed, next step - let's automation login functionality, verify credentials. Question is- why automation team is asked to take care of steps 2. and 3.. I am automating so that a.you don't have to test it test it manually b. you don't have to test for multiple user credentials.
    – paul
    Apr 14 at 13:28
  • 1
    Note: What @pavelsaman and I mentioned about the wording is a bit off-topic. E.g., you said "Test cases were written as per Acceptance Criteria", which dangerously assumes that the A.C. are complete (explicit x tacit knowledge) and its intention was perfectly transcribed into the code and testing. You said "all bugs that were logged during are closed", whereas you said previously "all bugs resolved", which are very different things. Two references about this are "Exploratory Testing 3.0" by James Bach and "Perfect Software" + "Exploring Requirements" by Jerry Weinberg . Apr 14 at 15:26
0

The basic goal for automation in your case seems to be take the load off QA. However, as pointed out the automation scripts have to be written again over a short period of time (unstable build). This is basically not effective, and will cost more in terms of investment. In my opinion, a better way to convince the client would be to stress on the point of bad investment if tests are automated at this stage as compared to when they will be automated once a more stable build is available. You can try to present some estimate statistics. The following qa testing companies links might be helpful,

https://blog.qasource.com/resources/-6-test-automation-best-practices-and-tips-for-startups

https://blog.qasource.com/4-essential-api-testing-automation-best-practices

-1

If you need to provide any kind of reference, I'd go with the syllabus for ISTQB Test Automation exam.

https://www.istqb.org/downloads/category/48-advanced-level-test-automation-engineer-documents.html

See chapter 1.1 Purpose of test automation

Objectives of test automation include: Reducing the total test cost

Automating a not stable app build can require additional effort/rework/maintanence, therefore from an economic perspective, this approach will be more expensive.

From chapter 6.1 Criteria for Automation

Suitability of automation for the stage of the software product lifecycle. An SUT has a product lifecycle which can span from years to decades. As the development of a system begins, the system changes and expands to address defects and add refinements to meet end user needs. In the early stages of a system’s development, change may be too rapid to implement an automated testing solution. As screen layouts and controls are optimized and enhanced, creating automation in a dynamically changing environment may require continuous re-work, which is not efficient or effective. This would be similar to trying to change a tire on a moving car; it’s better to wait for the car to stop. For large systems in a sequential development environment, when a system has stabilized and includes a core of functionality, this then becomes the best time to begin the implementation of automated tests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.