AI-based codeless automation tools are on the rise. There are plenty of them that even claim to have flexibility and reusability.

Are they good enough for large complicated test suites with many thousands of test cases?
Is strong programming skills not required for automation when using these tools?

Three examples:

Leapwork: https://www.leapwork.com/blog/codeless-test-automation
"codeless selenium with AI maintenance"

Perfecto: https://www.perfecto.io/codeless-automation
"AI-driven test automation creation"

Applitools: https://applitools.com/
"AI powered functional and visual testing"

  • 2
    Can you share some examples? Examples of those codeless automation tools as well as what you mean by "large and complicated test suites"? In general, tests should be simple in my opinion, or else you need tests for your tests, ... So no matter what tool you use, you should follow some practices that help you maintain your test suite(s) clean, easy to maintain, and readable for other people.
    – pavelsaman
    Feb 13, 2020 at 20:53
  • 1
    I added details to try and flesh out the question and address the valid issues from @pavelsaman Feb 13, 2020 at 21:20

2 Answers 2


In many cases, no

The first automation case is easy to write - no page objects, no need to use DRY, etc.

By the 100th test, you need a lot of programming - or codeless automation suites.

The problems really start when you then continue to use the

add or change functionality or bug discovered / fixed
write new UI test

The test suite starts to grow and before long takes first hours and then days to run. The number of test cases keeps growing (I've seen tens of thousands), as do the intermittent errors that destroy the confidence of the tests adding value. Multiply test cases by browsers and devices and versions and orientation and the Universe might end. Ultimately the original intent of rapid feedback disappears and QA becomes a separate function that does checking and verification but doesn't directly influence quality in the context of application code creation by developers. Testing gets done for the sake of testing and quality does not improve long term.

It also becomes apparent that you should be following the principles behind the Agile Testing Pyramid with the Unit, Integration and UI layers and just using a tool to write ever more UI cases does not promote this conversation and action within the organization.

Currently approaches such as 'keep the test suite under 15 mins by deciding which end to end UI tests are needed and which existing ones could be moved to integration or unit tests' are not approaches that AI test automation can handle as far as I'm aware. Maybe google have it internally.

  • "Maybe google have it internally" Sorry what? Feb 17, 2020 at 13:09
  • Maybe google have actually automated the process of deciding which end to end UI tests are needed and which existing ones could be moved to integration or unit tests' are not approaches that AI test. That is what I mean Feb 17, 2020 at 13:17

Here's my two cents.

I tried Applitools (the free account) and I think it offers some excellent features, like the speed of test creation and modification, less code written and ease of catching visual bugs. Another advantage compared to standard automation tests is the possibility of catching unexpected (visual) bugs. For example, you would usually write asserts for known web elements (that they are visible, contain the right text and so on) but you wouldn't usually assert that another element would become visible or that page layout would change.

Some of the downsides would be the price (obviously) and test speed execution since those types of tools need to upload screenshots to the server.

An approach that could benefit from visual testing would be to keep the usual test suites but maybe add a visual test case (one per page) that would check the page layout.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.