Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have been seeing a lot of customers pushing for browser compatibility testing using Selenium. Was wondering whether this is really necessary. The push is mostly due to Selenium grid supporting multi browser test execution in parallel.

As per my knowledge, Selenium is mostly a functional testing tool. So any issue that you find using Selenium will be across all the browsers. What additional value does it have if we find that the same issue exists in multiple browsers?

Or is there a better way in which Selenium can be used for compatibility testing?

share|improve this question

3 Answers 3

up vote 8 down vote accepted
  • Selenium solves a large part of cross browser testing. 90% I would say. Selenium is good for web testing across browsers. For UI related stuff adhoc manual checks are sufficient. I used to do manual checks to verify look and feel of UI (Text Sizes appearing small in a browser, Alignment Issues)

  • Functional UI testing in selenium across browsers verifies Id's exists in the UI, workflows-functionality are fine across browsers. Selenium cannot detect layout issues. You can custom code in selenium to detect java script errors.

  • There were few tools that I got based on my earlier question on layout bugs. Below tools you can use for Layout related checks and adhoc UI testing

share|improve this answer
1  
Thanks Alies Belik for formatting the answer –  Siva Mar 29 '12 at 16:53

Automated, parallelized, cross-browser testing is a seductive idea, but in my experience, the cost may not justify the benefit. I do not know whether your customer requests are based on actual experience or the advice of a trusted professional or the unfounded claims of someone in an elevator.

You may spend a lot of time maintaining your cross-browser tests. Even if you do not change your product, today's tests may work with today's browsers and break with next week's browsers. If you decide to leverage Selenium Grid, expect to spend a significant amount of time upgrading to new versions of Selenium and diagnosing whether the latest bug is in your test, in Selenium, or in your product. (Of course the bug may also be in the browser, but that is outside of your control.)

As @Siva mentioned, even if your functional tests run perfectly, you still have not verified browser-compatibility. No matter what, you still need to test manually.

You have a limited testing budget. The time you spend maintaining your cross-browser tests is time you cannot spend testing in other ways.

Still, your experience may be different from mine. I recommend approaching automated cross-browser tests as a limited-scope pilot project. Automate just part of the app and observe how much effort you spend keeping it running as the product changes and new browser versions are released. If you are satisfied with the expense, you can leverage technology more aggressively; or if you decide the benefit just not justify the cost, you can throw it away and think about something else.

share|improve this answer
    
+1, For ROI in practical scenario. Limited pilot project is a good start –  Siva Mar 29 '12 at 16:53

Selenium is mostly a functional testing tool. So any issue that you find using selenium will be across all the browsers

No, that's not actually the case. Some functionality can be broken in some browsers while working in others.

For example, for one of the apps I test, I've seen significant functionality issues going from IE 6 to IE 7 to IE 8.

share|improve this answer
    
I've also heard this across mobile platforms. Something that might work on an iOS device might not work on a BlackBerry, for example. –  joshin4colours Mar 29 '12 at 14:48
    
That's certainly true. Platform differences beyond just browsers can clearly elicit functional issues. –  Joe Strazzere Mar 29 '12 at 18:07
    
No wonder it's IE –  Tarun Mar 30 '12 at 4:13
    
Actually, we had issues when we expanded the supported platforms to include non-IE browsers, too. I just didn't mention that in my example. The world is divided into two camps - IE, and everything else. Issues arise on both sides... –  Joe Strazzere Apr 2 '12 at 20:33

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.