Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I am trying to create a suite of regression tests for an asp.net / javascript website.

I have been playing with selenuim & MS CodedUI tests but these all seem very complicated and more suited to a developer.

Is there a simpler tool that will allow me to take screen shots of sections of a page, and if that section changes, the test fails?

Is my idea flawed or incorrect?

Should I be asking how should I create robust and non fragile Website UI regression tests and what tools will help non developers create this.

The long term goal in the future one day, will be to create a robust and automated build service that can verify that the site is in a releasable state.

share|improve this question

2 Answers 2

up vote 3 down vote accepted

For really good, robust, maintainable UI automated regression, you need code. Using screenshots is bad for several reasons:

  • Even with a fuzz tolerance built in to prevent things like the exact color of the display throwing false positives, there are too many trivial things that can throw a false positive.
  • A change of display resolution will utterly destroy your regression.
  • You still have to interact with the application to get to the screens you're wanting to check. That's going to take either a lot of repetitive record/playback (bad) or well-optimized coding, particularly if you're dealing with the kind of application that doesn't do anything unless you're logged in with the proper credentials.
  • Screenshots don't tell you if the data your application is storing is correct.
  • For look and feel, there is no better tool than the human eyeball.
  • A human will always be better at determining whether a change to a screen is significant or not. A computer comparison algorithm will tell you that something has changed. It won't tell you if the change matters.
  • The UI of your web application is going to be different depending on the browser running the tests. Rendering of form fields varies dramatically between browsers. If you maintain a set of screenshots for one browser, you won't know if something has caused a rendering issue on a different browser. If you commit to automated testing against multiple browsers, you're talking about maintaining a massive amount of frequently changing screenshots. It's a maintenance nightmare.

Properly done automated functional regression is code.

That said, Selenium and CodedUI are probably not the best tools for someone with limited coding experience. The tools I know that could meet your needs as stated are:

  • SmartBear's TestComplete has what they call "keyword testing" which does most of the heavy lifting for you in terms of code. The licensing model is one of the cheaper ones for big box tools. There are a number of free to view webinars on the SmartBear site that demonstrate how the tool works (I am not affiliated with SmartBear in any way, but I used TestComplete for nearly 7 years).
  • TestAnywhere is an image-based tool that's reasonably sophisticated and can probably do what you want without overwhelming you. I personally found it too limited for what I needed, but then I'm comfortable with coding and have worked as a programmer in the past. It uses a hybrid of object identification and screenshots of individual fields to identify items.
share|improve this answer

IMHO it is not always a good idea create tests based on UI, they are really hard to maintain, btw for your case you can have a look at SIKULI, it is an UI automate testing tool, it is perfect for your goal.

share|improve this answer
    
Thank you for your reply. In your opinion, how would you ensure the UI has not been changed unexpectedly? –  SetiSeeker Apr 2 at 7:31
    
I've never had need to create UI regression tests, usually I do that kind of tests using link RobotFramework/Selenium and than I check the screenshot one by one. But in my case UI is not really important. cheers –  Onivas Apr 2 at 8:40

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.