Best approach to UI automation

Question

Let me start with my background,

Been in the industry for over a year, got the wonderful opportunity to join the team despite no prior background, loving and enjoying every moment so far actually. My only dilemma is that there were no senior QAs to pass down their knowledge to me.

Our teams are divided into frontend and backend, and I am part of the frontend team. As I mention, this is my first ever role, with no senior QAs passing down their knowledge. We have existing projects, but I felt like our automation weren't that great, in a sense that most of them are flaky, and I was fairly new to the role, so I didn't know much about automation strategy apart from the fact that - "Automation is good! You don't need to test things manually!"

After a while, I realise there is some truth to that, but that doesn't necessarily mean automate everything, which was my thoughts previously, to try to automate as much functionality each sprint.

Now we are starting a new project, and I am, again the sole QA for the frontend team. I would like to know how other teams approach their manual UI regression testing and also how you go about approaching automation

Manual: We all know that UI is very fragile, and there are lots of part and bits of pieces to it. From my experience, anything and literally ANYTHING could happen. So how do you guys keep track of every single thing?

Automation: From my learnings, it seems that UI automation are very flaky and should be kept to a minimum due to the always changing nature of the frontend. Since I am solely focusing on frontend, how would you guys decide on what to automate etc?

Any input would be appreciated. Would love to hear how other companies approach their frontend testing

Thanks in advance

Answer 1

For what to test use the Agile Testing Quadrants to divide up testing into unit, integrated, automated ui and manual ui.

For how much to test in any given area use the Agile Testing Pyramid (unit at the bottom, meaning the most, UI at the top meaning least) to generally guide the volume of tests in each area. The exact shape will vary at every company but generally it should be a pyramid. An hourglass shape is possible.

For

know how other teams approach their manual UI regression testing and also how you go about approaching automation

The key thing is to know how manual and automated testing will complement each other. Approach automation by educating yourself (I recommend the book Agile testing by Lisa and Janet) and then start having detailed and on-going conversations with the application developers. See a team effort with team decisions on what to automate where.

For the javascript components much of your unit testing will actually be ui testing by using nigthwatch. To understand what will need to be tested at a higher, perhaps end-to-end testing with selenium, get a good understanding of what is already covered by nightwatch. A frequent distinquisher is lots of different options for a given component that can be tested with nightwatch, leaving perhaps one end to end UI case to automated.

Answer 2

It will depend on what you mean by UI Automation.

If you mean whole system testing through the UI, indeed you have a manifold of moving parts that may change asynchronously:

• The UI itself
• The services your control
• The services you don't control
• Etc....

In this case two heuristics may help to keep maintenance sane:

• Execute frequently: In shorter time spans less things change, therefore maintenance time slows down;
• Small checks: If your tests depend on fewer components, less tests will break per change.

However, if you mean to test the UI (frontend apps), most of modern framework have components that allow low level checks, with tooling that leverages the framework context itself.

For instance, Ember.js allows you to test each component function as a unit, to build each component in isolation to test its rendering, or to build the whole Ember.js app to test the components integration, but mocking all services dependencies, making it less fragile to testing-unrelated changes in the services*.

// File: /mirage/config.js
this.get("/date", function(db, request) {
  // Mocking a JSON response from the /date endpoint
  return { day: "31", month: "12", year: "2018" };
});

React and Angular also have their own tooling.

These tests are focused on the UI** and, by removing the services dependencies, they are fast and reliable.

* The role of keeping the mocks in sync with changes in the service is of Contract Tests; * Meaning they can be added only to the UI building pipeline and failures will be due to changes on the UI code, only.