How can I successfully introduce automated user interface testing to my organization?

Question

Our development teams are good on writing unit tests and also for using the same framework (rspec in our case) for writing integrated tests.

There is a strong desire to increase automated UI testing. The desire has been expressed for the QA group to spearhead this.

When actually trying to do this however we have run into a lot of obstacles, such as:

• ui element tagging tends to be brittle. We decided that using data-attributes and specifically in our case data attributes that start with data-qeeg- to effectively namespace them to our QE Area for the eg application. We'd like to have full control within this name space but we are getting a lot of feedback from developers about the exact names we use.

• testers tend not to have some of the technical skills or experience that developers do. It's quite hard to find testers who are have a strong technical skills set in programming. this tends to perpetuate the (terrible) idea of testers as 2nd class citizens who are 'not good enough' to be the programmers they 'would really rather be'.

• duplication of effort. There is frequently push back from developers saying they have already written unit or even integrated (but non-ui) tests. Why do we need to duplicate a lot of their functionality?

• Co-ordination with developers. Having to follow all the same processes for change that developers use, does not seem to work well when we are making small changes to the UI to make automated UI testing easier. Developers have a tendency to want folks they work with to be skilled and experienced in other areas and can get frustrated with testers who are not. This seems to lead to a lack of trust and a 'always need to review it' mentality.

• Code reviews for element tag changes. The need for reviewing the code that is actually doing the automation seems reasonable and for that we fit into our organizations code review rules. However for the tagging of web UI elements that is an essential part of our work we are constantly slowed down by needing review and then often lengthy discussion of what are essentially naming conventions. We like to do peer review but when all our work if effectively dependent on 'programmer approval' it slows us down, demotivates us and makes us feel like 2nd class citizens.

• QE's have a role in each ticket. Manually testing tends to be quick and easy and fits within the times being spent on the ticket during its other phases. A problem arises when we want to automate the manual steps in that the automation takes a while to write and the 'needs testing' queue starts to back up and pressure build to just 'get the ticket out'

How can we overcome these obstacles and successfully introduce and grow Automated UI testing in our organization?

Answer 1

The issues in the question sound pretty typical of Selenium users and development/QA teams in general.

ui element tagging tends to be brittle.

Automated testing make the most sense when the interface stays the same but the implementation changes. If a page tends to change a lot, it may make more sense to test it by hand, or to test it with a combination of automated smoke tests and manual detailed tests.

By default, a developer is unlikely to care whether their UI breaks your automated tests, because they don't feel the pain of breaking those tests.

testers tend not to have some of the technical skills or experience that developers do.

I think that characterization is largely true. Most strong developers do not want to test, because testers tend to not be compensated or given the visibility that developers are. Salary surveys on job search sites like Indeed will back this up.

Of course, just because a tester is a below-average developer does not mean they can't be great at their job and effective at using tools like Selenium in appropriate ways. You need to find ways to let developers and testers take advantage of what they are already good at.

duplication of effort.

That is a fair question. Zero overlap is unrealistic, but UI test ought to focus on testing the UI, not on testing the business logic that's already covered by other kinds of tests. Your team needs to figure out what that means specifically for your application.

Co-ordination with developers. Code reviews for element tag changes.

It sounds as if developers need to own element tagging.

Manually testing tends to be quick and easy and fits within the times being spent on the ticket during its other phases.

When you write an automated test, you trade off risk and reward, just as you do when you decide whether/how to test something manually. You may need to evaluate whether you are automating the highest-yield targets.

In particular, "automate everything" is a strategy that leads to lots of high-maintenance, low-value code that does little to improve quality.

I think you should review what you are automating to convince yourselves and your developers that you are using automation in high-value ways. If you can't convince your developers that your tests have value, developers will not be inclined to help you.

Answer 2

ui element tagging tends to be brittle.

The tool you end up using will find elements in different ways. So you'll likely change up based on the tool you use. Hopefully the tool you pick makes it easy to change 'selectors' (find element logic).

testers tend not to have some of the technical skills or experience that developers do.

Look into using Telerik Test Studio. It can be used by the average QA person but it's functionality can be expanded upon with C#. There are many alternatives out there, but this is by far my favorite for entry level automation. It's similar to Selenium IDE in test creation.

duplication of effort.

Developers write unit tests that pass. That is not helpful to QA other than a Happy Path sanity check. Also, Unit/Integration Tests != UI Tests.

Co-ordination with developers. Code reviews for element tag changes.

Not all that necessary unless the element is too difficult to find using your selected tool. I have asked for the data-* attribute before, though.

A problem arises when we want to automate the manual steps in that the automation takes a while to write

You could test while automating, but that could be a waste if the ticket gets sent back and refactored. Make automation part of the process, same as Test Plans, and tell them to deal with it. Do they want it done fast or done right? In time you will find that automating is now second nature and a quick task. Again, that comes with time.

Answer 3

An answer to your second bullet point, from one project I've experienced recently:

Allow the QA department to draft developers as extra testers for UI tests. Allow the developers to spend some time on automating UI tests. Suddenly a lot of boring, repetitive tests got automated.

Make the testers/test analysts available to help with unit tests, especially the enumeration of negative test cases. They tend to be good at it, while many developers have blind spots.