Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

anti-pattern:

There must be at least two key elements present to formally distinguish an actual anti-pattern from a simple bad habit, bad practice, or bad idea:

  • A commonly used process, structure or pattern of action that despite initially appearing to be an appropriate and effective response to a problem, typically has more bad consequences than beneficial results, and
  • A good alternative solution exists that is documented, repeatable and proven to be effective.

I have recently encountered different anti-patterns in test automation, that make reading, reusing and maintaining tests hard.

A similar catalog has been created for anti-patterns in unit testing, but automation of end-to-end tests is different in nature from unit-testing. First, some patterns that are anti-patterns in unit testing, might be acceptable in end-to-end test automation. For instance, adding a new assertion to existing test instead of creating a new test case (The Free Ride / Piggyback design pattern) might be acceptable because setup in end-to-end environment takes usually more time. Second, there are anti-patterns specific for end-to-end test automation, e.g.:

  • Test data too much coupled with SUT DB
  • Environment configuration hardcoded in tests

Can you suggest others?

share|improve this question
3  
Looks like a good candidate for community wiki? I don't think there is one good answer for that. –  dzieciou May 4 at 19:18
    
Agreed on the cw suggestion - this is the kind of thing that would make a really good reference list because there are a LOT of possible anti-patterns around. –  Kate Paulk May 5 at 11:14
    
@KatePaulk, I can't change that into CW. Can you? I guess a question once upvoted cannot be turned into CW without moderator intervention. –  dzieciou May 5 at 19:20
    
I can't either - It must be a moderator thing –  Kate Paulk May 6 at 10:52
    
Brb, firing up the old CW machine from the storage closet. Haven't used this thing in ages, hope it doesn't fire at the wrong target. –  corsiKa May 6 at 11:56

5 Answers 5

Unit and system test automation is different, but at least a few of the unit test anti-patterns apply, such as concentration on happy-path scenarios. Thanks for including that link!

In the automation I have implemented, I was forced to implement GUI automation due to the structure of the legacy Java client-server systems I was testing. However, these are also applicable to web automation tools like Selenium that act at the user interface. Note that this applies to keyboard/mouse interfaces, touch interfaces are a different animal entirely and would require their own evaluation.

I will start with an obvious anti-pattern: Dependence on Record and Playback The alternative that I have implemented on all of my automation is to use recorded actions to obtain the structure of the GUI elements, then parameterize them into a function that searches the application interface structure to locate the object. This makes the GUI automation resilient to structural changes to the user interface (which happens often).

Making Intermittent Bugs Low Priority This will show up as "glitches" that a user or a manual tester would overcome quickly and often ignore. However, unstable or unpredictable operation is the bane of automated tests. Even if they are EBM ("every blue moon") intermittent defects, they would still prevent most automation from being implemented without a considerable number of restarts.

Here is my favorite: Ignoring Accessibility Development Standards These standards were developed to allow persons with disabilities access to software applications. Not only is it the right thing to do, but implementation of these standards improves the accessibility for all users. A little-known benefit of adopting these standards is the dramatic improvement in the testability of the application, both for manual and automated approaches. One aspect of these standards is especially important: making the application interface readable by external screen reader applications (such as Jaws). Not implementing accessibility standards forces you to implement "blind" automation, which is essentially the Record and Playback approach discussed above.

share|improve this answer
    
Jeff, can you elaborate more on how improving accessability for disabled persons improves testability of the product? Perhaps, with some example? –  dzieciou May 4 at 19:15
3  
For accessibility, the standards require that you provide a "accessible name" or "accessible text" ("alt text" in web apps) information that is passed to the external reader. Every tool I have used has leveraged that information if it exist. Another related requirement is keyboard accessibility. Having keyboard operation as an alternative to mouse clicks is a wonderful alternative automation method for GUIs that change their visual layout. –  Jeff_Lucas May 4 at 21:00
1  
An exception in software is not always a security issue. Exceptions are thrown all the time during runtime. Unhandled exceptions are bad, but not all pose a security risk. –  Bj Rollison May 4 at 22:10
1  
Driving automated scripts via key mnemonics is rarely a good practice, especially if you need to run your automated tests on localized environments –  Bj Rollison May 4 at 22:12
1  
An exception that produces a stack trace display at the GUI can be leveraged to create a SQL injection attack if the control codes are not intercepted properly in text field. –  Jeff_Lucas May 5 at 2:07

Not testing in Isolation Anti-Pattern

In order to run a test in isolation you often have to bootstrap one or multiple systems, configure these systems, setup test data, archive test artefacts and destroy the setup after the test run.

Easier is to just run the end-to-end tests against a some central system test database and shared server resources, with multiple accounts per test situation.

  • Dependencies on (production) infrastructure to run test suite
  • Parallel test runs could interfere each other
  • Test data is corrupted and needs manual fixing every now and then
  • Easier for test to become depended on each other

Better is to use a CI to use something like Vagrant to bootstrap a clean Test environment foreach individual test run.

share|improve this answer
    
So running a test in isolation is an anti-pattern, or boostrapping environment per each test is an anti-pattern or not running a test in isolation? –  dzieciou May 4 at 19:12
2  
Not testing in isolation is the anti-pattern and leads to all kind of problems. Often automated tests are not done in isolation because of the extra effort you need to setup a situation to run a test suite on its own. –  Niels van Reijmersdal May 4 at 19:15
    
Just for extra fun, sometimes it's impossible set up automated tests to run in isolation. This is my situation now: a web application which employs and relies on a number of web services that interface with a mainframe. The mainframe does not distinguish between "test" and "not test", and new (clean) data can only be entered via the mainframe - over which the web team has no control. –  Kate Paulk May 5 at 11:22

I'll take a stab at a few:

  1. Including business logic at the test case level instead of building a business layer. Existing pattern: Page Object Pattern
  2. Declaring page elements inline in test cases with locator information instead of in the business layer. Existing pattern: Page Object Pattern
  3. Sleeping for arbitrary amounts of time. Existing pattern: polling, explicit/implicit waits.
share|improve this answer
1  
Great! The last anti-pattern has been discussed in detail under a separate question: sqa.stackexchange.com/questions/3764/… –  dzieciou May 4 at 19:42

PageObject littered with Assertion anti-pattern.

Problem: Assertions are part of PageObject class

def varify_edit_link_presence
  //BAD: Assertion in page Object. 
  has_link?("edit").should be true
End

Solution: PageObject just provides status of the element to the caller. The caller itself will verify the status:

def has_edit_link?
   has_link?("edit")
End
  • Duplication makes life easy.
  • Long live dead code!!!!!
  • Rescue strategies.
share|improve this answer
    
What do you mean by rescue strategies? –  dzieciou May 3 at 6:34
    
Martin Fowler suggested to keep assertions out of PageObjects as well. But what do you mean by "long live dead code"? –  dzieciou May 3 at 6:41
    
The object which are in gonna for long period time for not accessing just remove that object its about rescue strategies. –  user3598383 May 3 at 11:52
    
some of the objects are not going for working so it takes a lot of load & create a confusion for assertion. some code are working as slowly but it is in used for long time period means "LONG LIVE DEAD CODE"..\ –  user3598383 May 3 at 11:55
1  
I corrected your answer a bit. Can you include your comments in the answer to explain bullet points? Can you use proper English as well? I'm sorry, but as it is now it is a bit hard for me to understand what you wanted to say in your comments. –  dzieciou May 5 at 8:40

Different Stack Anti-Pattern

The automated tests (also test frameworks) are implemented using a different software stack then the SUT uses. This will make it harder for developers to maintain the tests. I have seen web development teams needing to learn Java as an extra language, because the test team decided to us a Java testing framework without having the app developers involved. The testers where familiar with Java and not the PHP/JavaScript stack use by the other teams.

  • Demotivates developers to write, fix and or extend tests
  • Harder to maintain for a team
  • Extra setup to run the test local (Tools, Runtime, etc)
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.