Selenium seems to be the mainstay in automated functional UI testing. But I'm wondering what peoples thoughts are on the pros and cons of using Selenium?Have any of you chosen another route for automated functional UI testing? If so what was it and what were the reasons? If you do use Selenium, what benefits have you found?
Functional testing with Selenium? Can you be more specific?– Yu ZhangJun 13, 2017 at 21:38
1Functional UI testing.– CrustyeyelidsJun 13, 2017 at 21:40
- Open source, active contributions 24/7
- Wide range of supported languages
- Lots of online resources
- Good multi browser support / Parallel testing
- Excellent for what it does (when you learn to implement it properly to prevent flaky/brittle tests)
- Easy integration with the likes of Maven/TestNG/Jenkins etc
- Multi OS support
- Lightweight in terms of resourcing compared to some alternative tools
- Can be a tad harder to use than some other things, Selenium IDE isn't great
- Web based stuff only, e.g no windows applications
- Can not interact with everything in the browser, e.g some popout dialogs require additional handling elsewhere
- No official support channel
- Not ideal for image testing stuff
Do you know if the Selenium IDE is used that much? In particular by professional automation teams? Jun 13, 2017 at 22:38
@Crustyeyelids, no, Selenium IDE is not an ideal tool for large automation projects, as it is very hard to maintain.– Yu ZhangJun 13, 2017 at 22:49
Re: "Can be a tad harder to use than some other things,". What are some other things? Can you give some examples? Jun 14, 2017 at 9:03
Re: "Web based stuff only, e.g no windows applications". Well, it also does not test cars, not necessarily good for performance tests, but... well, it's a tool for testing web apps. Jun 14, 2017 at 9:05
+1 for best answer so far. Also don't forget: WebDriver might get standardized w3.org/TR/webdriver Jun 14, 2017 at 9:46
It is open source tool, anyone can download & use
It supports various operating environments (Windows, Linux, Mac etc...)
It supports various programming languages to create and execute Tests
It supports all popular web browsers
It supports parallel test execution
It uses less Hardware resources as compare to other popular tools like ...QTP/UFT
Since it is open source tool, No reliable Technical support (Official Users Group, Chat room in seleniumhq.org)
It doesn't support Desktop Applications/Windows bases applications
No Other tool integration for Test Management
Difficult to use as compare to other tools like QTP/UFT
It is a set of tools, e.g. JDK+Eclipes+Webdriver+TestNG+Firebug+Firepath so tool setup is more difficult than other tools like QTP/UFT where you configure one tool
New features may not work properly
Deployment of selenium is manual and thus more difficult than the UI of tools like UFT / RFT
Re: "It uses less Hardware resources". Less than what? Jun 14, 2017 at 8:59
Re: "Difficult to use". Comparing to what? That's very subjective. Jun 14, 2017 at 8:59
Putting "It's suite of tools" in cons for some people might be a pros. I actually like it, as it allows for more fine-tuning . Jun 14, 2017 at 9:03
Pros: you can start writing cross-browser code in your favorite programming language in a matter of hours
Selenium tests are unstable. WebDriver libraries version trail the auto-updating browser and there’s always something small that doesn’t quite work. Often when you get the new version of the Selenium libraries that is supposed to fix the issue, you discover that now something that used to work doesn’t work the same way anymore. Add to that multiple versions of browsers, and you are constantly chasing a moving target.
Good tests require good coders. Yeah, I’m sure there are plenty of good developers automating web testing through Selenium. But for everyone’s benefit it’s optimal to have your best developers write product features and leave automation testing to lesser skilled people. For better or worse, many companies leave UI test development to junior developers or non-CS graduates, which results in at best mediocre code quality that is a liability over time. Writing a good Selenium test requires understanding of the browser, CSS, AJAX and application page structure.
Test maintenance is a drag. This of course is true for any automated tests. The more of them you build, the more time you have to spend updating them for new features and dealing with false positives. The reason I hold Selenium culprit here is that it is a very low-level API that doesn’t offer much help to the developer in test maintenance like dealing with element locators, page screenshots and error capturing.
Selenium test suites take too long to run. Write enough Selenium tests and your suite will take days to finish. No framework out there can make a browser run faster and Selenium has pretty cool support for grid that can run parallel tests on multiple computers. But very few people have actually set it up and again, it requires quite a bit of skill to keep that beast running.
Selenium IDE lacks crucial functionality The concept behind the lightweight Selenium IDE is neat, but very few companies can actually use it for production purposes. The IDE doesn’t offer much in the way of functionality or browser support, and the tests it produces are, essentially, poorly implemented WebDriver code.
Low ROI Selenium is de-facto the only browser automation API that at a low level gives you everything you need. But it requires a very capable technical test automation engineer, and even then the productivity is low.
Please note: if you wish to reference an organization you are affiliated with, you must indicate your relationship with the organization or you risk having your posts removed as spam.– Kate Paulk ♦Jun 30, 2017 at 15:49
I must do it in my comment or in my user profile? Jul 3, 2017 at 5:46
In your answer, please. That way it's obvious to everyone who reads your answer that you are affiliated with that organization.– Kate Paulk ♦Jul 3, 2017 at 11:09
From the comment to answers, @Crustyeyelids, there is a big misunderstanding. There is a big difference between:
- "Selenium IDE" (small part of the Selenium project) and
- Selenium project with all the components: Grid, Webdriver, Selenium Server (and IDE).
Selenium IDE is almost useless, for real production tests.
Selenium (project) is W3C standard for browser automation. W3C standard is a BF deal. Google W3C to be enlighted.
Thanks. Yeah, my experience is mostly with the Selenium API. I've used the IDE one or two times, but never seriously. I wondered if anyone actually finds it to be a useful product or if it's just sort of sitting on a shelf collecting dust. Jun 14, 2017 at 19:34
So you've already been given some pretty good pros and cons, I'll answer your question regarding other possible routes.
I'm working with a company that bought several TestComplete licences, so I was asked to use TC instead of Selenium. While Selenium can be imported into TC, I found that you could create some pretty nice Object Model Pattern testing frameworks within TC without using Selenium - for both Web UI and Desktop applications.
It's much easier to set up page objects for sites that are text-field heavy. You can use the recording tool to click and enter text on every text box, and this then creates a script containing all of the text field accessors. I remember the days of using the Browsers developer console to copy them all out - and I don't miss it!
Ultimately, Selenium is great because it's quick to get started with. However, I'm creating tests around twice as fast thanks to the ability of scraping accessors from a website with TCs recording tool.
I dont think the ability to quickly locate elements determines how easy/fast it is to create tests. I would say that to do any non-trivial thing with testcomplete is a lot more complex than simply creating your own thing. Mainly due to available information on how to do it with selenium using popular programming languages vs how to do it with testcomplete. (whole internet vs just official wiki of a product). Not to mention transferable skills. (simplistic) programming vs using a product.– GeorgeJun 14, 2017 at 9:29
1Meh, don't really agree with this answer either. Plus, is a generated script always that clean? Jun 14, 2017 at 9:44
It's not clean when it's generated @FDM, I have to clean it up - but I essentially wrap them in methods inside a class. It's trivial, and quick to do. Plus, I would very much argue that a lot of the time IS spent in locating elements. The majority of Selenium questions on this site are related to locating elements. You can still write Python/Java code inside of TC too, so it's the best of both worlds. It's very much a viable alternative to Selenium. Jun 14, 2017 at 10:57
Yes, there are a lot of questions because people often don't know how to properly write CSS or XPath. Generated CSS or Xpath locators might possibly be brittle as well. Jun 14, 2017 at 11:17
It's as brittle as any other working CSS or Xpath locator. And if the developers change their naming schemes, it would only take a minute to fix each page. I'm not saying rely on the recording tool entirely, but I am saying TC is a valid alternative to using Selenium. Considering you can use what ever programming language you like in TC, and you can quickly create boilerplate code for tests, I see no reason why it's not an alternative. Jun 14, 2017 at 13:23