Asking UI developers to adjust markup to improve automated tests

Question

From time to time when doing test automation, a locator for the desired element is either too complicated, fragile or not readable. In cases like this, we ask developers to add IDs or other data-oriented attribute to an element.

The problem is, currently in this kind of situations the following thing happens:

1. a developer is asked to add an ID (or change markup in some other way) via instant-team-chat (e.g Slack)
2. the less desirable locator is added at the moment
3. a TODO comment is put near the locator to come back and use a better locator

And, you know what usually happens to TODO comments - they just stay in the codebase forever.

How would you improve the situation? Should we ask developers to improve markup for testers formally via Jira? Should we do it for every single element or somehow address it for the whole application at once?

Answer 1

I had exactly this discussion with our developers recently, and I do not have satisfactory solution.

My situation was complicated by the fact that (for our Angular-based pages) some tests are developed in protractor/javascript, and they can access specific to Angular, while vast majority of our e2e test are developed in Python/webdriver and cannot access those angular-specific locators.

First, disclosure: I avoid XPath locators like a plague. They are flaky, slow and brittle.

Adding IDs is not consequence-free: IDs have to be unique, and adding additional elements with same ID would break the old test which used it.

Name-based locators are more forgiving: if you have few elements with the same name (and you are prepared to handle the ambiguity), it is easier to recover by finding the right one from the list. IDs are often generated by widget itself, like some Angular/Material widgets, so adding a custom ID is not even an option.

Names work perfectly for things like rows, where every item in a column might have same name, and can be distinguished by some other attribute.

One solution (for lucky few) is to have a naming convention/best practices agreement with developers to add "name" attributes. If accepted, missing locator is breaking such agreement (definition of done), and is added on request.

Another solution (or for locators in pages which are not under active development, say if backfilling e2e test for existing old pages) is to have one "umbrella" bug to add locators, so developers can add them (and have a bug number to work with - in our process, even such trivial change has to have associated bug number).Such umbrella bug gives you a head start: it is already approved, scope and consequences are known. Any other bug would have to compete for resources.

OTOH you can just use CSS locators, which can use hierarchy of elements, but are more sturdy and faster than XPath. Of course some of such CSS locators would be long and complicated (worse than name-based locators), but you do not rely on co-operation (or lack of thereof) from developers, so you can advance faster, without waiting for developers and their goodwill.

This is all complicated by time pressure, and possible split between application developers and coders of automated tests. Because like automated testers, developers have their own priorities, and often making automated test coders more productive is not a metrics they care about (which relates to recent debate about different metrics while developing app and testing it).

Answer 2

My preference is to turn this into a team discussion with the goal of getting coding standards in place that require test-friendly locators to be used.

There can be resistance to changing the standards, but with a lot of tact and a bit of effort, testers can get this through.

The informal conversation is always my starting point. If that gets me nowhere, I start taking notes:

• Time to locate and test a usable locator for element X
• Time it would have taken to use an expected locator like ID
• Time for developer to add expected locator to code while coding
• Time for developer to retrofit locators to code after the fact
• Time to debug and then change to test code because of other issues with the undesirable locator and potentially.

What usually emerges is that it saves everyone time to do it properly the first time and put in unique identifiers on every element when they write the code.

Unfortunately, it's faster for developers to go with IDE defaults which don't necessarily add the identifiers you want or if they do, you get less-than-helpful defaults. In that case, it helps to be able to make the case to managers that the time developers save by not doing this is far outweighed by the time you lose trying to find a usable alternative - and that time is costing the company money.

Answer 3

Personally, I do not think making a request via Jira instead of via Slack would make any difference despite Jira being more formal and having more publicity and visibility.

My personal suggestion is:

• Invest some effort on looking into why developers only offer less desirable locators? Is there a different opinion regarding what a desirable locator is between testers and developers?
• The best way to catch less desirable locators is via code review among developers. Is it possible to introduce locator assessment into the existing code review process?

Should we do it for every single element or somehow address it for the whole application at once?

• It depends on the audience. If this is a general practice among several developers, it needs to be addressed as a whole. But if this is a relatively isolated incident, if I was a developer, I would like someone to come over to my desk and talk to me first.

Answer 4

I think it's a matter of how good relation in your team betwene devs and QAs in general, and between specific team members, in particular, is.

What helped in my team to improve collaboration and increase testability of the product (not only IDs for locators) was:

• Involving developers into writing new end-to-end tests and stabilizing existing ones. They decided to help when they found we have too many stories pending tests. This way they could share our pain and see more value from our tests.
• In return we try to review their component tests and propose new test scenarios for them. We were also open to some of their initiatives on how to stabilize end-to-end tests.

Answer 5

I think your problems lies in point 3 -- adding a TODO in the code. The best way to get rid of those, like to many other things, is to never introduce them in the first place. If you need the ID inserted pick a good ID from the start so you don't have to come back. Of course, that's not always easy so you might want to provide some standard way of naming things.

• Document it! The most important thing is to document that ID; make a task out of it and write the ID you came up with in that task -- if you think it may need revision, get someone to review it and note who helped you do the review. Slack is great for a number of things, but if it's an important part of your development process, you need documentation and Slack does not invite documentation.
• Review it! If it's hard to think of a name, you probably want to have a minor debate on the name. Set up a list of criteria that a name needs to meet in order to be accepted in a review. Make sure you review all IDs until people get the hang of naming things.
• Never add a todo! Seriously, it's not that hard once you get the idea of writing TODOs in the code out of your mindset. Documentation really is key in any kind of development of any kind and has been neglected for far too long by far too many people.
• Get rid of stale tasks. If you didn't do it and nobody complained in, say, 3 months, it was probably not worth doing anyway.

Speaking from the experience of a software developer who is currently tasked with updating the UI of a 30klos (kilo lines of spaghetti) component with no documentation, inconsistent naming and misuse of the chosen framework and the original developers having retired 10 years ago (i.e. an industry standard development job).

Answer 6

This has been a long running topic for me over many organizations.

Originally I hoped to be able to ask for or add myself the attributes that I wanted for automation. However I soon learned this would lead to too many unproductive conversations about the right names with developers.

Then I hoped to add data attributes for automation, having qe- attributes under my control. But this lead to duplication of effort and questioning of extra sized pages for end users.

Finally I've ended up with the conclusion that all the groups that edit the pages - application developers, UI developers and QE automation developers - all need to share the same UI and so need to share appropriate identifiers. That said, there still needs to be opportunities to communicate and the three that come to mind for me are code review, definition of done and development process communication.

I've also learned that there can be a fourth 'actor' - application frameworks. For example with ruby on rails the 'name' attribute is auto generated by rails based on database column name and for many fields will be an appropriate identifier to use.

Answer 7

How would you improve the situation?

Simple, add it yourself, that is what I do. Engineers who write test automation code are developers. Therefor I see no reason why the Test Engineers should not add the locators they need themselves to the front-end UI code.

The team should already have a process for code-quality (e.g. code-reviews). Any changes to the front-end code could follow the already used process to maintain code quality. It should not matter who adds the change, as I think the whole product team should practise collective code ownership. I would create a pull-request which includes my application changes and their tests, either let a developer review it or another test-engineer.

Should we ask developers to improve markup for testers formally via Jira?

Creating tickets in Jira feels like a major waste. Asking developers to do it on the-fly feels like it could negatively impact productivity, as context-switching is expensive. I would try to find a process where ping-ponging between test-automation and developer is eliminated. Discuss the options with the whole team.

Should we do it for every single element or somehow address it for the whole application at once?

No, YAGNI. If you want a better testable application from your developers I would ask them to create one happy-path Selenium test for each feature they deliver. Afterwards the automation team can add more cases and negative tests where necessary. Developers should know how un-testable the product is they deliver. I would say, "make it testable" is part of their job description!

Answer 8

Many have mentioned the teamwork part which would help, but frankly I think the best way to improve is to improve the locator method and not constrict dev more on labels. Get solid coding standards, but then work on making your locators more dynamic and relational instead of so static.

What I mean by this is Xquery which is designed originally for XML but ports completely to HTML logic structure. You can search within every part of the html structure including referencing parent/sibling relationships. This allows you to find an element even if it's dynamically moving around the page based on whatever in the application is consistent to pull from. With javascript you can bypass most of the browser parts and modify the HTML dynamically which can easily break static locators despite what values they have. With a single page application this makes all static ones easily broken, but if you can find a span where dynamic controls exist and then drill to the control type and check a displayed component of it then you can re-use the same xpath (xquery) to locate the exact element despite changing id's, css, values, names, structure, etc...

This requires testers to learn and understand the way the pages get generated and modified and then the locators relationaly scaffold to the dynamic components of the site rather than locking onto the specific elements at a point in time. Then runtime interpretation pulls the elements.

Naturally the methodical approach is application specific and not generic, thus anyone having issues doing this would need specific help instead of generic like this, however, the point is that let developers design what is best for the product so long as it's standardized and automation engineers learn to flow with the product. Sure doing id= is easier, but it doesn't make a better product, just makes it more static for testing. Some really cool javascript totally takes over the HTML making it virtually unpredictable during runtime, it doesn't make it bad code, QA needs to be adaptable. Check out D3, HighCharts, Angular, etc... work and look great but totally mess up static locators for automation. Dynamic automation to go with dynamic sites is a better way to go if you want to scale with developers instead of restrict them.