Is Selenium the right tool for browser compatibility testing?

Question

I have been seeing a lot of customers pushing for browser compatibility testing using Selenium. Was wondering whether this is really necessary. The push is mostly due to Selenium grid supporting multi browser test execution in parallel.

As per my knowledge, Selenium is mostly a functional testing tool. So any issue that you find using Selenium will be across all the browsers. What additional value does it have if we find that the same issue exists in multiple browsers?

Or is there a better way in which Selenium can be used for compatibility testing?

Answer 1

• Selenium solves a large part of cross browser testing. 90% I would say. Selenium is good for web testing across browsers. For UI related stuff adhoc manual checks are sufficient. I used to do manual checks to verify look and feel of UI (Text Sizes appearing small in a browser, Alignment Issues)

• Functional UI testing in selenium across browsers verifies Id's exists in the UI, workflows-functionality are fine across browsers. Selenium cannot detect layout issues. You can custom code in selenium to detect java script errors.

• There were few tools that I got based on my earlier question on layout bugs. Below tools you can use for Layout related checks and adhoc UI testing

  • Free Google tool: Fighting Layout Bugs

  • Useful Tools for Testing Cross Browser Compatibility

  • Browsera is dedicated provider dealing with cross browser testing

  • Top 10 Free Web Apps to Test Cross-Browser Compatibility

Answer 2

Selenium is mostly a functional testing tool. So any issue that you find using selenium will be across all the browsers

No, that's not actually the case. Some functionality can be broken in some browsers while working in others.

For example, for one of the apps I test, I've seen significant functionality issues going from IE 6 to IE 7 to IE 8.

Answer 3

Automated, parallelized, cross-browser testing is a seductive idea, but in my experience, the cost may not justify the benefit. I do not know whether your customer requests are based on actual experience or the advice of a trusted professional or the unfounded claims of someone in an elevator.

You may spend a lot of time maintaining your cross-browser tests. Even if you do not change your product, today's tests may work with today's browsers and break with next week's browsers. If you decide to leverage Selenium Grid, expect to spend a significant amount of time upgrading to new versions of Selenium and diagnosing whether the latest bug is in your test, in Selenium, or in your product. (Of course the bug may also be in the browser, but that is outside of your control.)

As @Siva mentioned, even if your functional tests run perfectly, you still have not verified browser-compatibility. No matter what, you still need to test manually.

You have a limited testing budget. The time you spend maintaining your cross-browser tests is time you cannot spend testing in other ways.

Still, your experience may be different from mine. I recommend approaching automated cross-browser tests as a limited-scope pilot project. Automate just part of the app and observe how much effort you spend keeping it running as the product changes and new browser versions are released. If you are satisfied with the expense, you can leverage technology more aggressively; or if you decide the benefit just not justify the cost, you can throw it away and think about something else.