1

I am having trouble choosing a good browser when automating a web application. Among the browsers I have considered are Firefox, Chrome, Safari, PhantomJS

I am particularly looking for comparisons of:

  • framework support, especially Selenium but also other frameworks
  • page/content loading time
  • execution speed
  • automation support
  • link navigation
  • opening other window or alert handling strategy

What are the advantages and disadvantages of different browsers with respect to these areas?

Currently I am using Firefox but I have observed issues like

  • overlong page loading time,
  • click does not always work when used with text/link text

I have not been able to find a simple, understandable comparison of browser support for Selenium and related automation tools particularly in terms of how well they handle automation of important features like link navigation, filling out forms, managing alerts or popup windows, and so on.

My project does not have any unusual functionality. I just want to be able to be able to build my automation cleanly.

Improve this question
10
  • 4
    Suggest you focus on figuring out which browsers your users are currently using and focus on that. Automating a browser... that no-one uses... is probably not a good use of time.
    – Michael Durrant
    Commented Jul 6, 2016 at 10:26
  • 1
    Voting to close as this will be primarily opinion based, because for some PhantomJS will be a dream come true because its fast for running tests. Some love Chrome for debugging, others might vote Firefox because its open-source. Question if you need to run your tests on more then 1 browsers else pick the one that works for your team. Personally I would start with Chrome.
    – Niels van Reijmersdal
    Commented Jul 6, 2016 at 10:28
  • You can use your server logs or a service like New Relic to see what browsers to focus on.
    – Michael Durrant
    Commented Jul 6, 2016 at 10:29
  • 3
    I've updated the edits to make this a less opinion-based question by asking for a comparison between major browsers instead of asking for the "best".
    – Kate Paulk
    Commented Jul 7, 2016 at 14:10
  • 1
    @Michael Durrant Then nobody would use phantomjs. There are other types of tests and other criteria than browser popularity and compatibility that make headless browsers useful for automation.
    – dzieciou
    Commented Jul 8, 2016 at 4:41

1 Answer 1

Reset to default
2

The choice of a browser depends on your goal. If you are testing GUI rendering and JavaScript execution you should always tests against those browser that your users/customers are using. So even if, for instance, Internet Explorer is slow and unhandy, it make sense to test against it, while if you don't find any bug with PhantomJS it does not mean much, because no real user is using it. Having said that, here are my experiences.

PhantomJS

  • I found PhantomJSDriver (called previously GhostDriver) particularly useful for testing SSL: checking whether a web site can verify a client certificate. With FirefoxDriver it requires pre-configuring a profile a certificate, with PhantomJS it is a matter of a few command-line parameters that can be passed at runtime.
  • The problem with PhantomJS (not the driver itself) is that it is released only once a year, so if you are waiting for a stable version with a bug fix, you need to compile a snapshot version yourself and it takes some time (20-30 minutes) to have it.
  • Some sources are claiming PhantomJS tests are significantly faster than in headful browsers, I haven't seen any credible benchmark proving that.
  • Finally, there are bugs that cannot be found with PhantomJS, see my another question: What bugs cannot be found with PhantomJS?.

HtmlUnit

  • Another headless browser if you don't need a JavaScript support. The fastest browser I am aware of.
  • Comparison with PhantomJS can be found here.
Improve this answer
4
  • 1
    Super @dzieciou - I too used PhantomJS and observed its good to have but real blocker is their slow pace deployment cycle :( Currently working with Firefox browser and heading towards satisfaction. You?
    – Narendra Chandratre
    Commented Jul 12, 2016 at 5:09
  • @NarendraChandratre I don't like headless browsers when it comes to tracking test progress (screenshots is not the same are seeing changes at real time) and here Firefox and Chrome win. PhantomJS is also not a real browser, though it is based on old WebKit version that makes it similar to Chrome and Safari, but still it does not make it 100% credible. Finally, I haven't seen any credible benchmark proving PhantomJS tests are faster than headfull browsers tests. My only reason to use PhantomJS for test automation stems from my current project needs (SSL).
    – dzieciou
    Commented Jul 12, 2016 at 5:31
  • (y) What you will say about FF & Chrome?
    – Narendra Chandratre
    Commented Jul 12, 2016 at 5:40
  • @NarendraChandratre AFAIK, Chrome does not offer a concept of profiles, always starts with bare basic configuration which makes some tests harder. E.g. Firefox offers a concept of profile with plugins for manipulating HTTP request headers, which I found useful in some tests but this makes then porting your tests to other browsers harder. I like Chrome for Developers tools that offer great troubleshooting support that beats Firefox with Firebug, but that is for manual testing.
    – dzieciou
    Commented Jul 12, 2016 at 5:47

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.