There must be at least two key elements present to formally distinguish an actual anti-pattern from a simple bad habit, bad practice, or bad idea:

  • A commonly used process, structure or pattern of action that despite initially appearing to be an appropriate and effective response to a problem, typically has more bad consequences than beneficial results, and
  • A good alternative solution exists that is documented, repeatable and proven to be effective.

I have recently encountered different anti-patterns in test automation, that make reading, reusing and maintaining tests hard.

A similar catalog has been created for anti-patterns in unit testing, but automation of end-to-end tests is different in nature from unit-testing. First, some patterns that are anti-patterns in unit testing, might be acceptable in end-to-end test automation. For instance, adding a new assertion to existing test instead of creating a new test case (The Free Ride / Piggyback design pattern) might be acceptable because setup in end-to-end environment takes usually more time. Second, there are anti-patterns specific for end-to-end test automation, e.g.:

  • Test data too much coupled with SUT DB
  • Environment configuration hardcoded in tests

Can you suggest others?

  • 5
    Looks like a good candidate for community wiki? I don't think there is one good answer for that.
    – dzieciou
    Commented May 4, 2014 at 19:18
  • Agreed on the cw suggestion - this is the kind of thing that would make a really good reference list because there are a LOT of possible anti-patterns around.
    – Kate Paulk
    Commented May 5, 2014 at 11:14
  • @KatePaulk, I can't change that into CW. Can you? I guess a question once upvoted cannot be turned into CW without moderator intervention.
    – dzieciou
    Commented May 5, 2014 at 19:20
  • I can't either - It must be a moderator thing
    – Kate Paulk
    Commented May 6, 2014 at 10:52
  • Brb, firing up the old CW machine from the storage closet. Haven't used this thing in ages, hope it doesn't fire at the wrong target.
    – corsiKa
    Commented May 6, 2014 at 11:56

7 Answers 7


Unit and system test automation is different, but at least a few of the unit test anti-patterns apply, such as concentration on happy-path scenarios. Thanks for including that link!

In the automation I have implemented, I was forced to implement GUI automation due to the structure of the legacy Java client-server systems I was testing. However, these are also applicable to web automation tools like Selenium that act at the user interface. Note that this applies to keyboard/mouse interfaces, touch interfaces are a different animal entirely and would require their own evaluation.

I will start with an obvious anti-pattern: Dependence on Record and Playback The alternative that I have implemented on all of my automation is to use recorded actions to obtain the structure of the GUI elements, then parameterize them into a function that searches the application interface structure to locate the object. This makes the GUI automation resilient to structural changes to the user interface (which happens often).

Making Intermittent Bugs Low Priority This will show up as "glitches" that a user or a manual tester would overcome quickly and often ignore. However, unstable or unpredictable operation is the bane of automated tests. Even if they are EBM ("every blue moon") intermittent defects, they would still prevent most automation from being implemented without a considerable number of restarts.

Here is my favorite: Ignoring Accessibility Development Standards These standards were developed to allow persons with disabilities access to software applications. Not only is it the right thing to do, but implementation of these standards improves the accessibility for all users. A little-known benefit of adopting these standards is the dramatic improvement in the testability of the application, both for manual and automated approaches. One aspect of these standards is especially important: making the application interface readable by external screen reader applications (such as Jaws). Not implementing accessibility standards forces you to implement "blind" automation, which is essentially the Record and Playback approach discussed above.

  • Jeff, can you elaborate more on how improving accessability for disabled persons improves testability of the product? Perhaps, with some example?
    – dzieciou
    Commented May 4, 2014 at 19:15
  • 3
    For accessibility, the standards require that you provide a "accessible name" or "accessible text" ("alt text" in web apps) information that is passed to the external reader. Every tool I have used has leveraged that information if it exist. Another related requirement is keyboard accessibility. Having keyboard operation as an alternative to mouse clicks is a wonderful alternative automation method for GUIs that change their visual layout.
    – Jeff_Lucas
    Commented May 4, 2014 at 21:00
  • 1
    An exception in software is not always a security issue. Exceptions are thrown all the time during runtime. Unhandled exceptions are bad, but not all pose a security risk. Commented May 4, 2014 at 22:10
  • 1
    Driving automated scripts via key mnemonics is rarely a good practice, especially if you need to run your automated tests on localized environments Commented May 4, 2014 at 22:12
  • 1
    An exception that produces a stack trace display at the GUI can be leveraged to create a SQL injection attack if the control codes are not intercepted properly in text field.
    – Jeff_Lucas
    Commented May 5, 2014 at 2:07

Not testing in Isolation Anti-Pattern

In order to run a test in isolation you often have to bootstrap one or multiple systems, configure these systems, setup test data, archive test artefacts and destroy the setup after the test run.

Easier is to just run the end-to-end tests against a some central system test database and shared server resources, with multiple accounts per test situation.

  • Dependencies on (production) infrastructure to run test suite
  • Parallel test runs could interfere each other
  • Test data is corrupted and needs manual fixing every now and then
  • Easier for test to become depended on each other

Better is to use a CI to use something like Vagrant to bootstrap a clean Test environment foreach individual test run.

  • So running a test in isolation is an anti-pattern, or boostrapping environment per each test is an anti-pattern or not running a test in isolation?
    – dzieciou
    Commented May 4, 2014 at 19:12
  • 3
    Not testing in isolation is the anti-pattern and leads to all kind of problems. Often automated tests are not done in isolation because of the extra effort you need to setup a situation to run a test suite on its own. Commented May 4, 2014 at 19:15
  • 3
    Just for extra fun, sometimes it's impossible set up automated tests to run in isolation. This is my situation now: a web application which employs and relies on a number of web services that interface with a mainframe. The mainframe does not distinguish between "test" and "not test", and new (clean) data can only be entered via the mainframe - over which the web team has no control.
    – Kate Paulk
    Commented May 5, 2014 at 11:22
  • @KatePaulk But the wen app talks to the mainframe through Web services, right? Can't you mock those Wev services then?
    – dzieciou
    Commented Mar 16, 2016 at 18:16
  • @dzieciou - nothing so advanced. The Web app "talks" to the mainframe through virtual punch cards in files dropped into designated directories. As far as the web app is concerned, data magically appears in the database and it drops files somewhere which are magically processed.
    – Kate Paulk
    Commented Mar 16, 2016 at 19:45

PageObject littered with Assertion anti-pattern.

Problem: Assertions are part of PageObject class

def varify_edit_link_presence
  //BAD: Assertion in page Object. 
  has_link?("edit").should be true

Solution: PageObject just provides status of the element to the caller. The caller itself will verify the status:

def has_edit_link?
  • Duplication makes life easy.
  • Long live dead code!!!!!
  • Rescue strategies.

This is an excerpt from the slide-deck on Agile Testing Anti-Patterns and Rescue Strategies. For more on this, you should look into that presentation.

  • What do you mean by rescue strategies?
    – dzieciou
    Commented May 3, 2014 at 6:34
  • Martin Fowler suggested to keep assertions out of PageObjects as well. But what do you mean by "long live dead code"?
    – dzieciou
    Commented May 3, 2014 at 6:41
  • The object which are in gonna for long period time for not accessing just remove that object its about rescue strategies. Commented May 3, 2014 at 11:52
  • some of the objects are not going for working so it takes a lot of load & create a confusion for assertion. some code are working as slowly but it is in used for long time period means "LONG LIVE DEAD CODE"..\ Commented May 3, 2014 at 11:55
  • 1
    I corrected your answer a bit. Can you include your comments in the answer to explain bullet points? Can you use proper English as well? I'm sorry, but as it is now it is a bit hard for me to understand what you wanted to say in your comments.
    – dzieciou
    Commented May 5, 2014 at 8:40

I'll take a stab at a few:

  1. Including business logic at the test case level instead of building a business layer. Existing pattern: Page Object Pattern
  2. Declaring page elements inline in test cases with locator information instead of in the business layer. Existing pattern: Page Object Pattern
  3. Sleeping for arbitrary amounts of time. Existing pattern: polling, explicit/implicit waits.

Anti-patterns in Test Automation

  1. Using implicit waits
  2. Using Recorded playback
  3. Using brittle, layout based selectors
  4. Not isolating test data between tests
  5. Not consistently using Page Objects
  6. Tests that only have negative assertions
  7. Using assertions mid-way through a test
  8. Repeating parts of tests due to the same setup
  9. Not considering the Sad path of user entering invalid data
  10. Forcing all tests to go through all pages in complex workflows
  11. Assuming that test automation cover accessibility and usability
  12. Using cryptic names and acronyms in test and suite descriptions
  13. Not considering optional workflows for both happy and sad flows
  14. Only testing 1 browser and assuming that other browsers work fine
  15. Not using Page Helpers to DRY up and name common code sections
  16. Using general assertions like have_css over declarative ones like have_button
  17. Using authenticate / authorization in multiple tests leading to massive test failures
  18. Bullet Points that gradually get longer as you scan your eyes down the full page of them

I would say that I've made every mistake in this list when starting out, before I knew better so I think they all meet your criteria of 'seemed ok initially'

  • 1
    Why is #7 a bad idea? If I have a complex test (for example: fill out a form, close the form, check if a form counter has gone up) I like to use an assert to make sure the form is closed before continuing. So I know where the test failed.
    – Mate Mrše
    Commented Aug 21, 2018 at 13:30
  • 1
    #7 is sometimes useful and valuable, particularly in complex multi-step integration tests: softwareengineering.stackexchange.com/questions/412980/…
    – Andrejs
    Commented Jul 30, 2020 at 9:43
  • @MateMrše the problem is that now you are testing ('asserting') something other than what the test was designed to test. Any carefully crafted error messages telling you about the unexpected outcome are often replaced. I do get the setup issues and have dealt with many myself. A good approach is to have multiple tests with each one stubbing out the dependencies to that point. A common example is when login fails. In some places this means hundreds of test failures. In others it means just one or two. This is very hard to do but it is the best way for success from what I've experienced. Commented Jul 30, 2020 at 9:49

Different Stack Anti-Pattern

The automated tests (also test frameworks) are implemented using a different software stack then the system under test (SUT) uses. This will make it harder for developers to maintain the tests. I have seen web development teams needing to learn Java as an extra language, because the test team decided to us a Java testing framework without having the app developers involved. The testers where familiar with Java and not the PHP/JavaScript stack use by the other teams.

  • Demotivates developers to write, fix and or extend tests
  • Harder to maintain for a team
  • Extra setup to run the test local (Tools, Runtime, etc)
  • I'd say there are valid cases for using this pattern: (1) C code is much easier to test with a C++ testing framework (2) Testing an API / executable file might be considerably easier with a different language (e.g. testing in Python while the executables's source is C++)
    – DarkTrick
    Commented Mar 14, 2022 at 5:13

White Box Testing In The Absence of Functional Requirements

Sometimes, lacking requirements, we read the system code to attempt to discern what it was supposed to do, and then write tests based on that knowledge. This can arise when someone is asked to automate tests for a legacy system.

There is a place for white box testing, but it is not a substitute for actually knowing what the system is supposed to do. White box testing without a knowledge of actual requirements also risks enshrining system logic errors in your tests.

An alternative (aside from getting some requirements) is to use comparator tests. You run two versions of a system using the same inputs and then compare the outputs. This kind of test is less precise than a functional test; if the outputs do not match, you know something has changed, but you do not know whether the change represents a new bug, a correct bug fix or intentional feature change.

  • Hmmm... But it's not specific for test automation, it's valid for testing in general, right?
    – dzieciou
    Commented Mar 16, 2016 at 18:15
  • Yes, that's right. When I read the question, I didn't get the sense it was soliciting anti-patterns specific to test automation.
    – user246
    Commented Mar 16, 2016 at 18:42

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.