Selenium pros and cons

Question

Selenium seems to be the mainstay in automated functional UI testing. But I'm wondering what peoples thoughts are on the pros and cons of using Selenium?

Have any of you chosen another route for automated functional UI testing? If so what was it and what were the reasons?

If you do use Selenium, what benefits have you found?

Answer 1

Pros:

• Open source, active contributions 24/7
• Wide range of supported languages
• Lots of online resources
• Good multi browser support / Parallel testing
• Excellent for what it does (when you learn to implement it properly to prevent flaky/brittle tests)
• Easy integration with the likes of Maven/TestNG/Jenkins etc
• Multi OS support
• Lightweight in terms of resourcing compared to some alternative tools

Cons:

• Can be a tad harder to use than some other things, Selenium IDE isn't great
• Web based stuff only, e.g no windows applications
• Can not interact with everything in the browser, e.g some popout dialogs require additional handling elsewhere
• No official support channel
• Not ideal for image testing stuff

Answer 2

Pros

• It is open source tool, anyone can download & use

• It supports various operating environments (Windows, Linux, Mac etc...)

• It supports various programming languages to create and execute Tests

• It supports all popular web browsers

• It supports parallel test execution

• It uses less Hardware resources as compare to other popular tools like ...QTP/UFT

Cons

• Since it is open source tool, No reliable Technical support (Official Users Group, Chat room in seleniumhq.org)

• It doesn't support Desktop Applications/Windows bases applications

• No Other tool integration for Test Management

• Difficult to use as compare to other tools like QTP/UFT

• It is a set of tools, e.g. JDK+Eclipes+Webdriver+TestNG+Firebug+Firepath so tool setup is more difficult than other tools like QTP/UFT where you configure one tool

• New features may not work properly

• Deployment of selenium is manual and thus more difficult than the UI of tools like UFT / RFT

Answer 3

Pros: you can start writing cross-browser code in your favorite programming language in a matter of hours

Cons:

1. Selenium tests are unstable. WebDriver libraries version trail the auto-updating browser and there’s always something small that doesn’t quite work. Often when you get the new version of the Selenium libraries that is supposed to fix the issue, you discover that now something that used to work doesn’t work the same way anymore. Add to that multiple versions of browsers, and you are constantly chasing a moving target.

2. Good tests require good coders. Yeah, I’m sure there are plenty of good developers automating web testing through Selenium. But for everyone’s benefit it’s optimal to have your best developers write product features and leave automation testing to lesser skilled people. For better or worse, many companies leave UI test development to junior developers or non-CS graduates, which results in at best mediocre code quality that is a liability over time. Writing a good Selenium test requires understanding of the browser, CSS, AJAX and application page structure.

3. Test maintenance is a drag. This of course is true for any automated tests. The more of them you build, the more time you have to spend updating them for new features and dealing with false positives. The reason I hold Selenium culprit here is that it is a very low-level API that doesn’t offer much help to the developer in test maintenance like dealing with element locators, page screenshots and error capturing.

4. Selenium test suites take too long to run. Write enough Selenium tests and your suite will take days to finish. No framework out there can make a browser run faster and Selenium has pretty cool support for grid that can run parallel tests on multiple computers. But very few people have actually set it up and again, it requires quite a bit of skill to keep that beast running.

5. Selenium IDE lacks crucial functionality The concept behind the lightweight Selenium IDE is neat, but very few companies can actually use it for production purposes. The IDE doesn’t offer much in the way of functionality or browser support, and the tests it produces are, essentially, poorly implemented WebDriver code.

6. Low ROI Selenium is de-facto the only browser automation API that at a low level gives you everything you need. But it requires a very capable technical test automation engineer, and even then the productivity is low.

Answer 4

From the comment to answers, @Crustyeyelids, there is a big misunderstanding. There is a big difference between:

• "Selenium IDE" (small part of the Selenium project) and
• Selenium project with all the components: Grid, Webdriver, Selenium Server (and IDE).

Selenium IDE is almost useless, for real production tests.

Selenium (project) is W3C standard for browser automation. W3C standard is a BF deal. Google W3C to be enlighted.

Answer 5

So you've already been given some pretty good pros and cons, I'll answer your question regarding other possible routes.

I'm working with a company that bought several TestComplete licences, so I was asked to use TC instead of Selenium. While Selenium can be imported into TC, I found that you could create some pretty nice Object Model Pattern testing frameworks within TC without using Selenium - for both Web UI and Desktop applications.

It's much easier to set up page objects for sites that are text-field heavy. You can use the recording tool to click and enter text on every text box, and this then creates a script containing all of the text field accessors. I remember the days of using the Browsers developer console to copy them all out - and I don't miss it!

Ultimately, Selenium is great because it's quick to get started with. However, I'm creating tests around twice as fast thanks to the ability of scraping accessors from a website with TCs recording tool.