12

I am currently researching testing frameworks to test C Sharp developed web applications. I have been leaning more towards the use of Selenium over Watir or Lightweight Test Automation but hesitant over the IDE being a Firefox plug-in as would then need to find a recorder for IE as that is the browser used to run the web applications.

Wondering as to any benefits that Selenium has over these alternatives when testing web applications created using C Sharp. Anyone with experience testing c sharp web applications with any advice on the best one to use.

I am also looking at testing WPF in the future and wander if Telerik’s Test Studio is best way to go about this.

4
  • I feel this falls under the "X vs Y" category and should be closed.
    – corsiKa
    May 20, 2011 at 16:28
  • 5
    As I understand it, the concern about x vs y is that people are likely to fall into camps if the question is wide open, plus the question of simply x vs y doesn't necessarily solve a real problem. This question has a lot of other details that affect whether x or y is more appropriate, and seems open to suggestions of z or w, plus is very clearly a real problem for which the OP needs a solution. I think those are key factors. May 20, 2011 at 17:49
  • @Ethel let's not forget the Selenium proposal was merged into this proposal. This means a significant portion of the users of the site are here for nothing other than selenium and are going to be naturally biased in favor of it. That lends me to think asking if Selenium is right is going to get little more than soul food.
    – corsiKa
    May 20, 2011 at 19:40
  • 1
    @glowcoder That doesn't seem to have been borne out in the answers since. I'm seeing balanced responses giving useful detail about what factors the responder considered.
    – testerab
    May 21, 2011 at 9:17

9 Answers 9

12

Users of the Selenium IDE will either use the Record-Playback mechanism or hand-code suitable statements to perform action and asserts. However this automation can only be used manually by a firefox user with the browser plugin. To move beyond this and have scripts that can be run on other servers and in other browsers, you should look at using Selenium Web driver which supports several browsers: chrome, IE, Firefox, opera, safari etc. Selenium web driver supports C# as well. Also you can setup Selenium Grid for remote execution of tests. Note that you can export seleniumIDE tests to the language of your choice.

You can use selenium IDE to understand commands if you are a beginner. If you select an element on Firefox browser and right click it would display all the commands related to the selected control in a context menu. It is mainly useful for beginners. Whenever you are stuck, you can quickly refer selenium IDE for guidance.

So, I would suggest you to go for Selenium Web Driver since it supports your requirement for IE and C#.

3
  • 1
    Agreed, this was how I got into using it and I've been able to make some good C# scripts from my recordings until I got comfortable with the structure and wrote my own.
    – MichaelF
    May 20, 2011 at 14:40
  • This looks promising, thank you for your advice.
    – Chris
    May 20, 2011 at 14:51
  • Updated to indicate that Record-Playback is NOT the only way to use the IDE and, as mentioned in many comments, most users of the seleniumIDE never use Record-Playback and hand-code scripts to work correctly. Other limitations such as only running in firefox and needing a human to run them are still true and limit its use. Nov 7, 2016 at 11:04
23

It's not actually a true "either this or that" question since you can actually use both together via watir-webdriver.

Firstly: If we are talking about testing at the browser level, you can pretty much completely ignore the language used to run the server side. There's no C# remaining once the page is rendered to HTML+Javascript and sent over HTTP to the browser. So there's no need to use c# as your test automation language. (you might still want to, but you don't NEED to)

Secondly Purge the idea of record/playback from your requirements. In the long run it just doesn't work for anything above the level of the HTTP protocol (recorders are fine at the protocol as a starting point in a loadtest scenario, but not for function testing at the UI level) This is something most experienced testers have known for about 15 years now and is an area of broad consensus among experienced QA people. The resulting tests are brittle and hard to maintain. The only people that like record/playback are salespeople of vendors of tools that depend on that level of tech who use them to convince managers that with their tool, "anyone can do test automation" IMHO the whole reason the Selenium IDE/Recorder exist is to compete with that mentality, few Selenium users I know use it, or the older Selenium Grid.

So what language do you use? With that in mind, use whatever works best for your testers, and dovetails with whatever frameworks you are using such as FITNess, or Cucumber etc. Personally I think that Ruby is an excellent language for testers to script in because it is easy to code and learn. Thus if your QA folks are not experienced coders, it may be a better way to go than trying to teach them C#. Also the Watir API is very object oriented and easy to understand for anyone that has grasp of HTML and the DOM (Domain Object Model) of a webpage. I'm also using cucumber to do BDD, which is done in Ruby so that dovetails well with Watir. OTOH if the people that will be doing this automation know another language, then you might want to go with Selenium RC and that language.

edit: The biggest factor in deciding what language to use often comes down to 'who do you want working on tests' If you are fine with silos and having just your testers or SDET type folks work on E2E automation, then pick whatever language works best for them. OTOH if you want your developers to be able to fix tests they are breaking with UI changes etc, then you may want to pick a language common to what your dev team is using. With agile teams using embedded testers as a part of the team, and not doing handoffs, that sort of thing can be a much larger concern going forward. Also there are now Java and JS versions of tools like Cucumber, so you have new options there these days as well. (this is something that has changed a lot in the 5 years since I originally authored this answer)

Enter Webdriver The thing is that Watir and Selenium are closer than many people realize, especially now that the Watir-Webdriver gem has arrived which uses Selenium's new Webdriver technology (which as I understand it, is also replacing Selenium 'Core') to drive the browser under the hood, but allows the tester to write code in ruby with the excellent Watir API. With both Watir and Selenumn moving to Webdriver for interfacing with the browsers, it appears to be the future of both projects. Using Watir-Webdriver gives you IMHO the best of both worlds, with wide browser support due to Webdriver and the easy coding of Watir.

I'd try the tutorials of both (Selenium RC and Watir) and see which API seems like it would be easier for you to use for coding your tests. If you like Watir better, be sure you immediately adopt using it with Watir-Webdriver as well.

2
  • +1 for your remarks on record/playback. I really wish vendor salespeople would keep their mouths shut on that front. They're tricking IT managers into thinking it's a sustainable solution :( Nov 8, 2011 at 20:38
  • Another +1 for rejecting record/playback. Feb 5, 2012 at 18:19
7

I'm afraid I'm not very familiar with Watir, but I do have some experience with Selenium 1 and 2.

I've used Selenium RC on a Python project before, and really loved it. But rather than RC, I have been using Selenium 2 WebDriver for testing a C# developed application. I started out using Selenium RC, but quickly switched because I found I needed support for things like OnPageLoad popups, which Selenium RC doesn't offer.

The Selenium 2 C# assemblies are really rich with extensible features, but there are a few notable items from Selenium 1 that are missing from 2, such as WaitForPageToLoad(), and IsElementPresent (which I was able to reproduce using a method constructed from FindElement).

On the IDE: I've not found it very helpful at all, even when I was using Selenium 1 / Python. The scripts it generates are really generic and poorly constructed, and needed alot of refactoring. So, if you go the RC route, you're better off just coding everything yourself, and using the IDE for "suggestions", if you get stuck on something.

Hope this helps.

2
  • I've found it useful a few times, I don't know why so many people harp on the IDE it's a tool like any other. Either you have use for it or you don't. I agree the WebDriver is awesome and was a great addition when I was writing C# libraries for testing.
    – MichaelF
    May 20, 2011 at 16:54
  • If you're using Selenium 2, it's a moot point, because the IDE generates Selenium 1 code. But I agree that, if you're using RC, it can be helpful as a reference/teaching tool. However, I wouldn't rely on it for my finished product, because the output isn't very idiomatic. May 20, 2011 at 17:48
6

For C# development, you can use either Selenium RC or WatiN. Watir adds the need to learn Ruby, which might be nice for companies using a LAMP stack.

Our team had several developers look into Selenium, WatiN, and Watir for our C#-heavy project, and concluded that Selenium might be better if you had fewer testers with extensive development skills. This is mainly due to the record-and-playback functionality in Selenium being greatly superior to what exists for WatiN. Selenium also had the advantage of being more common and easier to find information for. However, WatiN had richer libraries, "felt" nicer to code for people used to C#, and was liked better by the developers. Both Selenium and WatiN work in both Firefox and IE with relatively little effort, but WatiN's way of handling multiple browsers felt tidier from an object-oriented perspective.

tl;dr: I'd use WatiN if your testers tend to also be developers, but Selenium if development time that can be dedicated to testing is at a premium and you need to minimize the coding burden. Both work well w/ C#, IE, and FF.

2

If you have some scripting skills in Ruby I would by pass the record/playback and code the test directly in WATIR http://www.watir.com/

1

I am actively using Telerik Test Studio at the moment. It is a pretty powerful record and playback tool and long with a pretty solid framework that is similar in structure to WatiN and Selenium RC. With Test Studio the underlying test framework is free but the record and playback studio piece is not. It has plugin integration with VS 2008 and 2010 which makes it really nice when you want to add some coded steps into a record and playback piece. You can evaluate it for free however so you might take a peek at it if you are interested. I have not used it yet for WPF testing.

1

For C# testing you should use WatiN instead of WatiR. Also consider using of Telerik Test Studio. It has some benefits for test automation.

1

Have your looked at VSTT - Microsoft Test Tool. VSTT 2010 should be a good bet if you are looking for functional test automation. Web Services Testing, UI testing, BizTalk testing and Data Driven Testing Support.

Please look at

0

IF you app is developed in C# why wouldn't you use C# to test the app? C# has many useful libraries such as Reflection "to see into the app," and you virtually eliminate any abstraction layer between the product code and automated tests.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.