Need to make WatiN work for us, or need to find a different tool

Question

tl;dr version: WatiN Test Recorder looks dead, and I am unsure of the status of WatiN itself. I need to make them work or find alternatives that meet our requirements. Suggestions?

Longer version:

Our tester has a long list of manual regression tests for our internal, line-of-business web applications. I am trying to help with automating these tests so that they can be run on demand or at the end of each automated build.

We are a .NET shop for web applications, and the internal web applications are IE-only. I don't get to change either of those facts.

Given the above, it seems that our requirements for a testing framework are:

1. Needs to record and run tests in IE
2. Needs not to involve coding when recording the tests (the other .NET developer and I can and will need to do some minimal coding to create test suites, handle any dynamic data, etc.)
3. Needs to handle some Javascript, AJAX, etc.

Strong preferences include:

1. Code for running the tests can be written in .NET, and the tests can be integrated into Visual Studio and/or MSBuild projects. Java is a distant second choice, and anything else would be very disadvantageous. I know other languages, and I believe the other developer does also, but when we hire developers later, fewer language requirements would be a good thing.
2. Open source tools that work are great. Commercial tools are certainly not out of the question, but we do need to limit the cost, and the tool definitely needs to be worth the cost. I don't know exactly where the line is drawn, but I've been told that the Rational suite is likely too expensive to be approved, for instance.
3. An active community using the tool and good documentation would be nice. A tool that is continuing to be developed and improved is also a good thing.

Initially, it appeared to me that WatiN would meet our requirements nicely when combined with WatiN Test Recorder. That was before I tried them. I only tried the 2.0 beta for Test Recorder, but there seem to be several problems still (even once correcting the inability for the stock installation to run on 64-bit Win7).

I remember watin.org (down at the moment, though the problem seems to be under investigation) at least implying that there were multiple test recording solutions, though WatiN Test Recorder was considered best, but I have been unable to find them. Are there any?

Finally, if I cannot make these tools work for us, what are good alternatives that meet the requirements I have stated?

Answer 1

The key question here is, "Why are you using Record and Playback techniques, rather than coding your tests using Page Objects?" I can think of two reasons:

1. Your team lacks the technical skill to code tests in C# without using Record and Playback.
2. You just didn't realize that there are other ways to create automated UI tests that are more robust and much, much easier to maintain (and faster, after a little initial effort). However, you have testers who can write quality code without relying entirely on record-and-playback.

If your team fits into category #1, your best option may be to record everything you can in Selenium using Firefox, then use IE developer, some common sense, and maybe some help from developers to fill in the bits that don't work in FireFox. Then you can set the automation to run in IE (I'm afraid I don't know how, but I've done this before), and actually run your automation against the correct browser. Be aware when doing this that Record and Playback testing like this is intrinsically fragile and difficult to maintain, and choose the areas of the UI that you will be testing wisely.

If your team fits into category #2, you have a much better option, which Dmitry gave in his answer. This solution - coded Page Objects - works with either Selenium RC or WatiN. Testers who don't also think of themselves as developers tend to prefer Selenium, possibly due in part to the ability to use Record and Playback to get hints about how to develop, while developers tend to like the rich library of WatiN. Selenium probably has the stronger support in the tester community.

Here is a discussion of the Page Object design using WatiN. You should also read the post about attaching a Browser and the FindBy attribute. Dmitry has good links for Selenium.

Answer 2

I have moved on to Selenium and the WebDriver for unit tests. I have not tried to record anything in the FireFox IDE as I am comfortable with writing tests in JUnit.

http://seleniumhq.org

Answer 3

First of all, please take a look on the following great blog post:

Keep Your Tests DRY

The post refers to the Page Object pattern that helps to make you code clean. I am doing automation on WatiN and using this pattern instead of step by step recording and copy pasting the clicks on the user login screen.

IE Developer – that’s all what I need.

Of course, WatiN is good only if your application should supports IE and/or Firefox.

If you want a framework with support for all modern browsers – please, consider Selenium.

Answer 4

If the automation needs revolve around automation and IE one option is Visual Studio Ultimate, which does have record and playback with some options for coding, so while the Tester gets used to the interface eventually there is the same environment for coding the tests in C# or even using the CodedUI modules to creature reusable functions/modules.

You don't mention cost, and VS Ultimate is not cheap unless you have the right MSDN subscription, so I put it out there as an option I am using.

In the past I have also used Visual Studio (not Ultimate) to write tests using Selenium, recently I have converted over to the Selenium 2 webdriver and it's pretty slick.