Tell me more ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.
up vote 3 down vote favorite

I came across the ApprovalTests.com framework today and am curious how it would be classified. I am familiar with concepts of Unit Tests, functional tests, and user acceptance testing, but this seems to be a different kind of testing as it is based on matching test outputs against pre-approved outputs that have been verified or generated by a human. I want to call it verification or validation testing or maybe automated User Acceptance Testing, but I am curious if there is an official term for a framework that uses previously approved output values instead of just assertions on primitive types.

Also are there any other frameworks that use a similar approach? It appears that ApprovalTests has integrated well with Eclipse/Visual Studio, data serializers/viewers, and various Diff tools, but I would be interested in seeing if and how anyone else has done this type of testing.

share|improve this question
Something inside me says {automated} and {user} are mutually exclusive. It might be automated acceptance testing, but not automated user acceptance testing. Still seems appealing though. – corsiKa♦ Jul 25 '11 at 18:05
@glowcoder Hmm... that is a good point. I like sound of Automated acceptance testing. Looks like there may not be a formal definition for this, but I'll keep my eye out for anything else that looks similar. – Greg Bray Jul 26 '11 at 0:15
There's a lack of formal definitions for most of the SQA field, unfortunately. – corsiKa♦ Jul 26 '11 at 16:18
Most definitions are localized versions, depending on the company, country and people's experiences. Most of what is in the Approval Tests framework looks to me just like validation, it could be in any kind of test type/phase though. You could code these things, or automate input/outputs in almost any framework. – MichaelF Jul 27 '11 at 9:03

3 Answers

up vote 2 down vote accepted

There is a concept called the 'Golden Master' testing which is what approval tests uses, and tries to automate.

The reason that the concept is orthogonal to Unit Tests, functional tests, and user acceptance is that all of these are still doing verification by asserting against primitive values.

All tests have 2 parts:

1) do

2) verify

The real difference between those tests is the scope of the 'do' part. Unit tests are small, usually 1 or 2 methods. Functional tests run end to end, usually through a dependency border (server, database, etc...) and user tests always go through the 'front door' of a GUI (or the like) to do things.

Approval tests are all about the 'Verify' side of things. As such can be used for all 3 types of testing, and more (like system characterization testing). Approvals allow for you to easily verify at bigger levels, but that does not mean that scope is not important, it just means that you can now choose without the paying a higher cost in verification if the scope is bigger.

Hope this helps

share|improve this answer
Thanks. Looks like a great testing framework. I'll have to give it a try on my next project. Keep up the great work! – Greg Bray Jul 31 '11 at 18:10
up vote 3 down vote

Approvaltests.com calls them unit tests, so I'm not going to argue :) Although I'm not sure why you are looking for a hard definition. It's taking the value from unit tests, adding readable output and screenshoting them. Kinda a neat process that could be used for UAT and forces dev's to use TDD. Tools that I would consider similar are Cucumber, jBehave, and Concordion (each has thier own specific flavor though).

share|improve this answer
up vote 0 down vote

I would call Approvaltest as Regression_tests: it verifies that the result of a test is the same as the last execution of the test.

In other words it verifies that the result hasnot changed.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.