5

From time to time, our end-to-end tests fail because of a markup change made by one of our UI developers - for instance, there was an id on an element but it got removed/replaced, or an Angular binding of an element changed, an element was moved to under a different parent etc.

The problem is that, UI developers don't really know what, we, testers use to locate elements on a page and, hence, they don't keep automated tests in mind when they are changing the markup - it is quite easy for them to break our locators which results into us debugging and changing the location strategy which is generally "expensive".

Is this a communication issue? Should we somehow establish markup change rules? What other ideas do you have to improve the process?

Improve this question
3
  • 1
    Using selectors with the minimum amount of information to uniquely identify the element and Isolating responsibility by following the Page Object Model and good programming practices has made markup changes not very expensive to debug and fix for most teams I've worked on. Maybe 1 hour a week for around 20 developers. That being said, if they are changing things like crazy and you have many dozens of tests to fix every few days something would need to be done about that. It doesn't hurt to have discussions about id's and automation with developers however.
    – mrfreester
    Jul 31, 2017 at 17:02
  • 2
    Fwiw, I don't think this applies to your situation, but I've heard some teams have the developers own the selenium tests. Using some kind of incentive, if their tests start failing, they have a limited amount of time to log a bug or fix the test before the test gets dropped from the suite. I've never personally worked with a company where the devs were also primarily responsible for the individual automated tests.
    – mrfreester
    Jul 31, 2017 at 17:04
  • @mrfreester yes, we are using Page Objects and try to be smart about the locators we use. But, the changes are frequent at this period of time which made me think to ask this kind of question here. We are also a little bit understaffed and that may make our situation a bit different from an average one. Thanks so much, good points, I will take them into an account.
    – alecxe
    Jul 31, 2017 at 17:36

2 Answers 2

Reset to default
5

Yes, this is a communication issue.

If any of the tests are failing this tells you that you have a change in the code that is not handled or you have a bug.

You should know that some attributes are also used for styling the page, maybe there is a good reason for changes.

To prevent this kind of issues you can do few things:

  • you should follow best practices to gets the selectors as efficient as possible to prevent any of these thing
  • your team should know the kind of work the automation team is doing like the automation strategy and best practices and this should be mentioned in a meeting and also kept in a wiki page
  • the automation team should keep a close relationship and communication with the all the other teams and also with UI devs to be ready for the changes that are coming
  • try to involve the dev team in the automation process if possible

If let say the UI devs are changing things with no best practices rules in mind then this it should be discussed in a meeting along with the side effects results like:

  • automation suite will have a lot of fails
  • the team cannot rely on automation suite for results
  • a lot of time is invested in fixing failed tests due missing of a UI guideline
  • the efficiency of the QA automation team will be affected and as a result you might have also delays in new builds since the time is not invested for writing new tests to increase coverage or for other tasks
  • there is no point in having automation if you cannot rely on it

Remember that the work should be done in a team.

You need to identify the issue and ask a PM / TL / QA Lead / QA Manager with this if needed.

Improve this answer
1
  • +1 for: "there is no point in having automation if you cannot rely on it" - because quality cannot be "assured" after the fact, it has to be build in, by the whole team, not only by QA group. Quality cannot be bought, but has to be paid for
    – Peter M. - stands for Monica
    Aug 2, 2017 at 15:39
2

My approach to this is generally to let the UI group runs the tests themselves. This way if they break a test they will know about it and have a chance to fix it.

There are several key factors to help make this work:

  • UI devs work in branches just like app devs
  • End to end Test Suite stays under 10 minutes
  • Use of a continuos integration server like jenkins or circleCI to run tests efficiently
  • Recognize that sometimes new or very different tests will be needed and that app or QE dev should be involved in writing the bigger change to the test(s)
  • Spot new tests and potential issues up-front during ticket triage and make appropriate notes for UI to know about
Improve this answer
1
  • 2
    While the first part can be considered as a possible answer, I think that the most of the list items doe not apply the question topic ui changes, but indeed they are very good guides to follow while doing automation.
    – lauda
    Aug 1, 2017 at 21:38

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.