How can I structure Selenium tests in a way that minimizes the maintenance work?

Question

What is the best way to build/structure the tests in a way that will minimize the work needed to keep the tests up to date as the product changes and improves?

Answer 1

As others said, use Pageobject and other best practices. Few links:

• PageObject explained - by Martin Fowler, language-independent
• Selenium Best practices - with short summary, for Java

Summary:

• Use PageObjects pattern
• Be fluent with
  • return this, varargs, generics,
  • reuse your model and jodatime
• Be robust and portable
  • Prefered selector order : id > name > css > xpath
  • Avoid Thread.sleep prefer Wait or FluentWait
  • Use relative URLs
  • Don’t rely on specific Driver implementation
  • Create your dataset
• Know your new tool
  • Keep up to date (versions and usage pattern)
  • Troubleshooting
    • jre 1.6
    • IE (zoom, Protected mode setting )
    • Firefox/firebug startpage
  • How to deal with UI components like... fileupload, datepicker, ajaxtables,...
  • Detect when selenium isn't a good tool for the job
  • Don't be afraid to hack around selenium

Googling "pageobject" and "best practices" will give you many more links for your system and language.

PageObject encapsulates a bunch of page interactions and locators for them, so the test can call a pageobject method to perform some action on a page without knowing about HTML, locators etc.

Answer 2

In addition to the awesome suggestions others provided, I would like to describe one often neglected area. Unlike in unit tests, in end-to-end tests, you have less control over the data in your system and over your test environment. Here are a few suggestions to make your tests easier to maintain when such data or test environment change:

• Decouple from existing test data. Don't assume your test data will exist forever in DB. I've seen tests that rely on production DB data, e.g., users with specific names or privileges. If references to those data are hardcoded and data behind those references will be updated or, even worse, removed, then the tests will be useless. Instead, make your tests create specific test data or retrieve them in a declarative way, e.g. through SQL queries.

• Name your test data. If you don't decide for retrieving or creating specific test data as a part of your test, give them at least some meaningful names that will explain why you picked up this sample and not the other. Imagine you have to maintain the set of test data containing flights that got outdated: no longer there are flights from Los Angeles to Reykjavik. So you need another sample of similar flights. What do similar test data mean here? I.e. What was the purpose of testing for this specific flight? Was it about checking system behaviour for an intercontinental flight or perhaps for a roundtrip? A better way would be to wrap this test data with a name, e.g., anyIntercontinentalRoundtripFlight().

• Decouple from test environment. Don't make your tests coupled to a specific test environment. I've seen tests with URLs, paths, DB connection URLs and other environment-specific elements hardcoded. When your product will have to be tested on a new configuration, updating your tests will be a nightmare. Hence, separate test environment configuration from your test cases. Put environment configuration in separate classes or files.

Answer 3

Readability: One of my code mentors taught me early in my career as: "If your logic/algorithm is not simple enough to understand and modify by a third person, refactor it. Rinse & repeat.

Down the line after 6 months, that third person might be yourself."

The few rules I personally follow while designing automated tests:

• Simplify your code: Simplify your code up to such extent that it naturally flows down from one layer to other and readable by third person. Can an non-technical person read & understand your code like a book?(at least on top layers)

• DRYup your code: DRY-up your code so if anything changes, you have to update at one place only. Always keep on refactoring the code to make it more DRYer.

• Design robust and flexible locators: which are short, unique but flexible enough to be hard to break

• Small focused simple tests instead of long complex tests: Have only one assertion as the last step in a test.

• Make tests independent of each other: No test should fail just because any other test failed.

• Every test should test only one thing: Every test should fail just because of one reason.If any test can fail because of multiple reasons, it is not properly designed.

• No redundant steps: There should not be any same steps/verification points covered in multiple tests.

• Decouple locators & test data: Isolate locators and test data from main automation code so it can be independently updated without touching the code.

Answer 4

Handle test just as your other code, use coding guidelines, development patterns, best practises and document methods, etc... keep it as maintainable as possible from a code perspective.

To keep the tests up-to-date I would add the following steps to your process:

1. Have the developers run the tests before they check-in/commit their code (make them responsible for not breaking automated tests)
2. Run the tests with a continuous integration server on each check-in/commit (so you know who/what broke the tests)
3. When a test fails identify if its a defect or if the test needs updating
4. Make sure someone is working on fixing the test before you continue any other development

The key is to give the developer the fastest feedback as possible and prevent building on quicksand.

Answer 5

Basically, you want to isolate the contents of the tests from the actual page interaction so that you can update the page interaction code once and have all tests working, right?

That's what the PageObject design pattern was invented for. If you have an object that knows how to interact with your site, and all your tests interact with that objects instead of touching the page directly, once the site changes, you can simply update the PageObject and all your tests will be up to date.

Answer 6

I am sure you must already know this- It is generally considered a bad idea to automate features/functionalities/modules which are likely to change in the future. But I know that sometimes it is not under anyone's but the client's control what happens to the product.

I have till now only worked on the data-driven framework and this is what I think you can do to minimize efforts in case the product changes-

1. Modularize your code. Create a base class which contains methods dealing with the most common/most used actions eg. Button clicks, context clicks, assertions, waits etc. This will encourage re-usability of the code in the Test class.
2. Get your test data from external files. You could also store the element identifiers in a file. This will help you change your test cases(to a limited extent) without having to re-compile your class/classes.
3. Conform to coding standards. If you have a large team and multiple QAs contributing to the automation process it is very important that everyone write their code in a proper format. This will enable every member of the team to be able to read, understand and change(if required) any part of the code, saving time and efforts
4. Use proper Framework. Using a framework like TestNG allows you to take/define a lot of actions apart from your test steps eg. which cases/classes to run/not run, which cases to execute first, which cases to be made dependent on other cases/skipped if one or more cases fail, etc

These are all I can think of right now. Hope it helps, but in my experience if what you are automating changes frequently, you end up spending a lot of unnecessary effort in trying to cope with the changes. Too many changes often lead to new issues being introduced in the code.

Answer 7

As Yash suggest but in short form:

Use Testng annotations which signals dependences quite good.

It tooks some time to learn use it good(yaml + groups + packages included), but for annotation you could build dependencies and structures of tests quite easy.

External property file is also helpful for user data.

Also use scripting language if allowed, it minimalize boilerplate code , simplify understending and writing hussles, and also gives chance to learn scripting language. If possible use DSL.