6

I work with Selenium for quite a while now and was very happy with it for some time. However recently, I get more and more projects where I see extensive use of JavaScript. For example webpages written with ExtJS. As a result, the DOM structure of the webpage becomes unreliable for automation purposes and it gets harder and harder to make it work with Selenium (or any other DOM based automation tool).

Now, given the fact that the above-mentioned technologies become more and more popular and relatively simple HTML webpages are harder to find, what does that mean in terms of automation tools like Selenium? As far as I see it, if that trend continues, and I see no reason why it wouldn't, will DOM based tools become useless? Are there any other tools that somehow interact better with JS based webpages?

Will be great to know some thoughts and\or personal experiences on that topic.

Improve this question
3

Eugene S, I had a similar problem with testing UI generated by ExtJS library.

The solution I found useful was to create separate classes for more complex visual components. For instance, for data grid components I have created ExtJsDataGrid class that let me pick up a specific cell in the grid, abstracting me from obscure details in Selenium (dynamic locators, JavaScript invocations, etc.).

There are even companies that develop libraries of such wrappers, backed with Selenium, e.g. InfoStretch did it for ExtJs. It would be great, though, if each company that release a new library for UI rendering to provide also a library in Selenium to test their components. ExtJs would provide library for testing ExtJs components, Angular for Angular stuff, etc.

Improve this answer
  • Hi and thanks a lot for your valuable input! Do you know where can I find any examples of such classes/wrappers? Will be great to have some reference. – Eugene S Sep 4 '14 at 17:06
  • I only know one. For ExtJs check InfoStretch: infostretch.com/QA/selenium-framework.php. This is commercial library in Java, released without sources, but you will be able to see interfaces and their methods to get an idea. – dzieciou Sep 4 '14 at 17:09
  • Also see my question for a sketch of the solution: sqa.stackexchange.com/questions/8882/… – dzieciou Sep 4 '14 at 17:47
2

I don't see DOM based tools going away until DOM itself goes away. In general this is getting more and more complex with graphics, animations, tricks to increase speed, etc... but the underlying structure of the browser and html is the same. The tricky parts are really just timing as the DOM is modified based on user interactions to avoid page reloading. For Selenium and other DOM based tools that simply means careful precision of steps to ensure that elements exist correctly before interacting with them. Technically this just enforces strong test step creation which is a good practice anyway.

I would see more support for html5 specific items like canvas and possibly even support for other built in browser capabilities, but frankly alot of these don't deal with the DOM at all as you mentioned. Two things stand out to me about this technology.

  1. The common theme in all of this is images. That being said I think the future is heading towards image recognition and testing based on the "eye" of the user. I think Sikuli is ahead of it's time and is an amazing step in the right direction. True there is still alot to be done, but I see more of a merging of DOM and image recognition for the future of test automation. Straight image recognition is cumbersome and difficult to maintain as well as taking up alot more disk space. DOM is much easier to deal with and closer to standards, but is not adaquate for all situations so it will be interesting to see how far this will go.
  2. Essentially though this is the exact same problem introduced with Flash and will most likely be handled in similar fashion. Flash automations "read flash" by the traces they leave. Sequences of behavior, results, properties, etc...thus what is being used is automatable by reverse engineering the existing technology. Most of all of this is done via javascript. Therefore rather than replacing DOM tools I would guess that people will come up with clever ways to execute javascript in order to mimic the existing user interactions with javascript. Thus your Selenium scripts will include more javascript execution instead of straight DOM execution.

I think both are viable. With all the developers though I think we will more than likely go with option 2. I would love to see option 1 get poured into more as that field will work on anything and will be more stable going forward as technologies change. However, it's also the more tricky and cumbersome of the two programming wise.

Improve this answer
  • Hi and thanks for your answer. When I mentioned DOM getting less reliable, I meant things like page elements get different names with every time the page is opened(rendered). That means I can't "attach" my driver to something stable. Can't agree with you more about Sikuli. I am using it quite extensively for more than a year and super happy with it. However, as you also mentioned, there are some limitations but then what tool doesn't have any? :) – Eugene S Sep 4 '14 at 16:34
  • There are dynamic mapping capabilities built into Selenium that allow associated attaching which can work around alot of what your are talking about. Not sure if you have tried that, but if you have an example that you "can't" do with Selenium, it might be worth posting as a question on here as I do stuff like that all the time in Selenium. – mutt Sep 4 '14 at 16:40
  • That sounds interesting. It will be great if you could share some links pointing at this topic! A good example of a scenario that can get very painful is when there is an element that changes it's ID every time page is loaded and it's the only unique way to identify that element. The only way I found to work through cases like this is to create offsets based on another element. But that approach is essentially unreliable by itself. – Eugene S Sep 4 '14 at 16:48
  • 1
    here is a link kind of going over simple situations. seleniumtests.com/2011/10/… In general this allows alot of complex relationship xquery to ensure the element is indeed the one associated with certain things on the page. This depends alot on the page structure and following what things get generated together and then going from there. (i.e. this title will have this text somewhere underneath it and there will be two controls following that text...etc) Goes up and down html depending. I assume you utilize xquery and not just xpath... – mutt Sep 4 '14 at 17:06
  • Actually I have never utilized xquery in my tests. I will definitely have a look into that. Thanks for that input! – Eugene S Sep 5 '14 at 2:25
1

I think Sikuli approach (image/pattern matching with customizable similarity) is the best solution when it comes to visual intensive content as in big data applications, I am currently working with it on both web and desktop application test automation and I find it a great tool, am importing it in my java classes and creating test cases and suites as I wish. They just need to finish handling OCR/text searching. Generating native mouse and keyboard events from java/Sikuli is very powerful and can handle both web and desktop applications seamlessly, that simply makes it a single tool that you can depend on for your test without complications of learning new technologies whenever a new development technology changes (and it will never stop changing), simply (what you see is what you test). Having multiple screen resolutions or even multiple states of the same element can be handled by asserting for a set of images instead of only one, simply you have the full power of java in your hand with the added power of image/pattern matching by Sikuli.

Update: I successfully got sikuli running in headless mode (no physical monitor connected)

configuration: windows 7, install display driver from virtualbox guest additions and use TightVNC to remotely set resolution from another machine.

enter image description here

more detailed answer here.

Improve this answer
  • Sikuli is a great tool indeed, however, as I have already mentioned, it has many limitations. The biggest one is that it can't work in a headless mode. – Eugene S Sep 11 '14 at 8:01
  • Headless mode can be achieved using virtual machine or VGA terminator. – Amr Lotfy Sep 11 '14 at 8:18
  • Is that something you personally have been successful with? If so I will be super happy if you could share your experience? According to official sources Sikuli won't even start in true headless mode. Reference: answers.launchpad.net/sikuli/+question/239356 – Eugene S Sep 11 '14 at 8:40
  • I have successfully tried Sikuli on oracle virtualbox/win7. – Amr Lotfy Sep 11 '14 at 9:56
  • Yes, but I assume you had your screen attached right? You didn't run it completely headless. Headless means no screen attached at all. Like a virtual system on a cloud for example. – Eugene S Sep 11 '14 at 12:52

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.