Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

What is the best way to build / structure the tests in a way that will minimize the work needed to keep the tests up to date as the product changes and improves?

share|improve this question
    
possible duplicate of Building "slow to break" regression tests –  dzieciou Jul 12 at 11:15
    
Can we merge this question and its answers with sqa.stackexchange.com/questions/4/… covers? The other has more answers but this one has a clearer title, IMHO. –  dzieciou Jul 12 at 11:15
    
Are either actually good questions? "best" is aribitrary. Between the two questions there's 18 answers, and a lot are radically different in approach. I'm not sure either version of the question is answerable. –  corsiKa Jul 14 at 19:19

6 Answers 6

As others said, use Pageobject and other best practices. Few links:

Googling "pageobject" and "best practices" will give you many more links for your system and language.

PageObject encapsulates bunch of page interactions and locators for them, so test can call a pageobject method to perform some action on a page without knowing about HTML, locators etc.

share|improve this answer
    
Those are very useful links. It would be great if you could summarize them, as Help section on SQA suggests: "Always quote the most relevant part of an important link, in case the target site is unreachable or goes permanently offline." –  dzieciou May 2 at 17:29

In addition to awesome suggestions others provided I would like to describe one often neglected area. Unlike in unit tests, in end-to-end tests you have less control over the data in your system and over your test environment. Here are few suggestions to make your tests easier to maintain when such data or test environment change:

  • Decouple from existing test data. Don't assume your test data will exist forever in DB. I've seen tests that rely on production DB data, e.g., users with specific names or privileges. If references to those data are hardcoded and data behind those references will be updated or, even worse, removed, then the tests will be useless. Instead, make your tests create specific test data or retrieve them in a declarative way, e.g. through SQL queries.

  • Name your test data. If you don't decide for retrieving or creating specific test data as a part of your test, give them at least some meaningful names that will explain why you picked up this sample and not the other. Imagine you have to maintain the set of test data containing flights that got outdated: no longer there are flights from Los Angeles to Reykjavik. So you need another sample of similar flights. What similar test data mean here? I.e. What was the purpose of testing for this specific flight? Was it about checking system behaviour for an intercontinental flight or perhaps for a roundtrip? A better way would be to wrap this test data with a name, e.g., anyIntercontinentalRoundtripFlight().

  • Decouple from test environment. Don't make your tests coupled to a specific test environment. I've seen tests with URLs, paths, DB connection URLs and other environment specific elements hardcoded. When your product will have to be tested on a new configuration, updating your tests will be a nightmare. Hence, separate test environment configuration from your test cases. Put environment configuration in separate classes or files.

share|improve this answer

Handle test just as your other code, use coding guidelines, development patterns, best practises and document methods, etc... keep it as maintainable as possible from a code perspective.

To keep the tests up-to-date I would add the following steps to your process:

  1. Have the developers run the tests before they check-in/commit their code (make them responsible for not breaking automated tests)
  2. Run the tests with a continuous integration server on each check-in/commit (so you know who/what broke the tests)
  3. When a test fails identify if its a defect or if the test needs updating
  4. Make sure someone is working on fixing the test before you continue any other development

The key is to give the developer the fastest feedback as possible and prevent building on quicksand.

share|improve this answer
1  
+1: Postponing maintenance of tests makes their maintenance harder, as the problems becomes bigger and bigger. –  dzieciou Jul 12 at 9:06

I am sure you must already know this- It is generally considered a bad idea to automate features/functionalities/modules which are likely to change in the future. But I know that sometimes it is not under anyone's but the client's control what happens to the product.

I have till now only worked on data-driven framework and this is what I think you can do to minimize efforts in case the product changes-

  1. Modularize your code. Create a base class which contains methods dealing with the most common/most used actions eg. Button clicks, context clicks, assertions, waits etc. This will encourage re-usability of the code in the Test class.
  2. Get your test data from external files. You could also store the element identifiers in a file. This will help you change your test cases(to a limited extent) without having to re-compile your class/classes.
  3. Conform to coding standards. If you have a large team and multiple QAs contributing to the automation process it is very important that everyone write their code in a proper format. This will enable every member of the team to be able to read, understand and change(if required) any part of the code, saving time and efforts
  4. Use a proper Framework. Using a framework like TestNG allows you to take/define a lot of actions apart from your test steps eg. which cases/classes to run/not run, which cases to execute first, which cases to be made dependent on other cases/skipped if one or more cases fail, etc

These are all I can think of right now. Hope it helps, but in my experience if what you are automating changes frequently, you end up spending a lot of unnecessary effort in trying to cope with the changes. Too many changes often lead to new issues being introduced in the code.

share|improve this answer
    
Keeping test data files in external files is not always a good idea for maintaining your tests. If you need to refactor your data structures, e.g., rename a field or add a new one, you will need to do it manually for each test data file, unless your IDE supports refactoring across both your programming language and your test data description language. –  dzieciou May 1 at 16:06

Basically, you want to isolate the contents of the tests from the actual page interaction so that you can update the page interaction code once and have all tests working, right?

That's what the PageObject design pattern was invented for. If you have an object that knows how to interact with your site, and all your tests interact with that objects instead of touching the page directly, once the site changes, you can simply update the PageObject and all your tests will be up to date.

share|improve this answer

As Yash suggest but in short form:

Use Testng annotations which signals dependences quite good.

It tooks some time to learn use it good(yaml + groups + packages included), but for annotation you could build dependencies and structures of tests quite easy.

External property file is also helpful for user data.

Also use scripting language if allowed, it minimalize boilerplate code , simplify understending and writing hussles, and also gives chance to learn scripting language. If possible use DSL.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.