Future of DOM based UI testing

Question

I work with Selenium for quite a while now and was very happy with it for some time. However recently, I get more and more projects where I see extensive use of JavaScript. For example webpages written with ExtJS. As a result, the DOM structure of the webpage becomes unreliable for automation purposes and it gets harder and harder to make it work with Selenium (or any other DOM based automation tool).

Now, given the fact that the above-mentioned technologies become more and more popular and relatively simple HTML webpages are harder to find, what does that mean in terms of automation tools like Selenium? As far as I see it, if that trend continues, and I see no reason why it wouldn't, will DOM based tools become useless? Are there any other tools that somehow interact better with JS based webpages?

Will be great to know some thoughts and\or personal experiences on that topic.

Answer 1

Eugene S, I had a similar problem with testing UI generated by ExtJS library.

The solution I found useful was to create separate classes for more complex visual components. For instance, for data grid components I have created ExtJsDataGrid class that let me pick up a specific cell in the grid, abstracting me from obscure details in Selenium (dynamic locators, JavaScript invocations, etc.).

There are even companies that develop libraries of such wrappers, backed with Selenium, e.g. InfoStretch did it for ExtJs. It would be great, though, if each company that release a new library for UI rendering to provide also a library in Selenium to test their components. ExtJs would provide library for testing ExtJs components, Angular for Angular stuff, etc.

Answer 2

I don't see DOM based tools going away until DOM itself goes away. In general this is getting more and more complex with graphics, animations, tricks to increase speed, etc... but the underlying structure of the browser and html is the same. The tricky parts are really just timing as the DOM is modified based on user interactions to avoid page reloading. For Selenium and other DOM based tools that simply means careful precision of steps to ensure that elements exist correctly before interacting with them. Technically this just enforces strong test step creation which is a good practice anyway.

I would see more support for html5 specific items like canvas and possibly even support for other built in browser capabilities, but frankly alot of these don't deal with the DOM at all as you mentioned. Two things stand out to me about this technology.

1. The common theme in all of this is images. That being said I think the future is heading towards image recognition and testing based on the "eye" of the user. I think Sikuli is ahead of it's time and is an amazing step in the right direction. True there is still alot to be done, but I see more of a merging of DOM and image recognition for the future of test automation. Straight image recognition is cumbersome and difficult to maintain as well as taking up alot more disk space. DOM is much easier to deal with and closer to standards, but is not adaquate for all situations so it will be interesting to see how far this will go.
2. Essentially though this is the exact same problem introduced with Flash and will most likely be handled in similar fashion. Flash automations "read flash" by the traces they leave. Sequences of behavior, results, properties, etc...thus what is being used is automatable by reverse engineering the existing technology. Most of all of this is done via javascript. Therefore rather than replacing DOM tools I would guess that people will come up with clever ways to execute javascript in order to mimic the existing user interactions with javascript. Thus your Selenium scripts will include more javascript execution instead of straight DOM execution.

I think both are viable. With all the developers though I think we will more than likely go with option 2. I would love to see option 1 get poured into more as that field will work on anything and will be more stable going forward as technologies change. However, it's also the more tricky and cumbersome of the two programming wise.

Answer 3

I think Sikuli approach (image/pattern matching with customizable similarity) is the best solution when it comes to visual intensive content as in big data applications, I am currently working with it on both web and desktop application test automation and I find it a great tool, am importing it in my java classes and creating test cases and suites as I wish. They just need to finish handling OCR/text searching. Generating native mouse and keyboard events from java/Sikuli is very powerful and can handle both web and desktop applications seamlessly, that simply makes it a single tool that you can depend on for your test without complications of learning new technologies whenever a new development technology changes (and it will never stop changing), simply (what you see is what you test). Having multiple screen resolutions or even multiple states of the same element can be handled by asserting for a set of images instead of only one, simply you have the full power of java in your hand with the added power of image/pattern matching by Sikuli.

Update: I successfully got sikuli running in headless mode (no physical monitor connected)

configuration: windows 7, install display driver from virtualbox guest additions and use TightVNC to remotely set resolution from another machine.

more detailed answer here.