Automating visual web site comparison

Question

A company that maintains several websites for various small businesses is migrating all of the sites over to new servers (they are using some sort of migration tool). They have asked me if I could write something that will visually compare each page with its migrated page to make sure they VISUALLY look exactly the same and if there are differences then flag those pages.

I have an idea how to do this but am open to other suggestions if anyone has done this before. My thought was to input a list of every home page and use a sitemap generator to get a url of every page under each site. Then I will screen capture every page using a headless browser like phantomjs. Next I will load a host file with the names/ip of all of the migrated sites and screen capture all of the new pages (same URL's just on different ip's). Finally I will use some image comparison code (maybe wraith) to compare each screenshot with its counterpart and flag ones that are different.

Seems like it could work but I would greatly appreciate others input. ((They also want me to automate form testing but I will create a separate post for that.))

Answer 1

You can use - Sikuli!

Its screenshot comparison and handles false negatives very well.

Answer 2

Wraith can take screenshots from the different versions of a website, lying upon each screenshot and show differences. - https://github.com/BBC-News/wraith

Answer 3

I have pretty good experience with the open source T-Plan VNC Robot or their commercial T-Plan Robot Enterprise version.

VNC Robot allow you to automate any computer system that has a VNC server installed. It finds and compares full or partial screenshot and you can use the supplied scripting language (with IDE) or Java to automate the tests and interact with the system keyboard and mouse interface. You should be able to automate the forms also with this testing application with ease.

The maintenance of tests that use screenshots is a bit harder, due to the fact that if the interface change you need to re-capture all or most of the screenshots.

In your case I wonder if automation is really the correct path to follow, since the start-up of getting an automated framework up and running costs a lot of overhead of you do not repeat these tests over and over again. If its a one time comparison, maybe doing it manual is more cost effective. See this question for comparing web-site screenshots for a number of tools you could use for this.