How to automate look-and-feel testing in a complex web application?

Question

My company's web application has dozens of distinct pages that a user might hit. There are site-wide stylesheets, so changing one of them could affect every single page.

There are four or five different login types, each receiving a distinct set of pages.

There are many different possible device sizes, so we use media queries to (try to) make every page responsive and work properly at any size.

But the combination of (login types) x (total number of pages) x (device sizes) is huge, meaning that it's impractical to manually view every one after making a change to a global style that could potentially affect any of them. And indeed, after making such a change, there is often a layout bug--and sometimes a really catastrophic one--because we failed to catch the issue by viewing the affected page(s) on a device size where the layout is messed up.

Is there a tool that could automatically run through a massive number of possible page types, download each one, and validate that the appearance is correct?

This seems tricky, because "correct" is a human judgment about whether the page looks right, or whether everything that's supposed to be visible is actually visible. Also, much of this content depends on API calls initiated by javascript, and layout steps that only happen in javascript, and the layout can depend on what is contained in the data returned from the API call.

Can this be done?

Answer 1

There are visual testing tools (such as Applitools, but I'm sure there are other cheeper options) that will take a baseline screenshot of the page and then will test against it.

You can validate the look (either pixel-by-pixel, or just layout of the page, and some more options) and do that for a variety of screen sizes, browsers, etc.

Answer 2

The term you are looking for is visual regression but what it does is , it validates the image is exact match or close to the reference image ( as you can choose match threshold) , As it is matched with reference image css style change will fail the test because the image will ofcourse change so you cannot use that

So in this case exhaustive testing is not possible the things you should ensure are :

1. Actual work flow is functional , meaning check the actual use cases eg: in a shopping website user able to buy a product and images are loading
2. This where BDD is important , you test the user behavior than the UI implementation. THis takes away focus from UI and puts it on the user.
3. Ensure you have adequate unit/component testing
4. Have compulsory Exploratory testing before the release. Ensure critical and sanity tests are working fine.
5. Have beta releases where possible to selected bunch of customers
6. Don't test on all devices , find the device with major user base and test only those and mention that in supported list. Address bugs as when reported
7. Make each team member to take the responsibility of testing a feature or page (All members not only QA)
8. At last remember that quality is a team effort and ongoing activity. Its about learning from results. See how product works where is defect clustering happens, like which device fails the most etc and increase the test activities on those devices.