Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I don't want to describe the domain I work in at my work, so I will describe some fake scenario; might be silly but hopefully illustrative enough. Let's say we have a system that qualifies whether an employee deserves for promotion or not and want to test it.

The system recognizes such candidates for promotion by counting how many hours in a row a user did overtime, how many commitments it completed. It does not operates on statistics, but do calculate them themselves based on the work registered by the user.

I could set data for test in a very precise way:

createUser()
  .with(overtime().on(DayOfWeek.FRIDAY).from(16).to(20))
  .with(overtime().on(DayOfWeek.SATURDAY).from(10).to(15))
  .with(overtime().on(DayOfWeek.SUNDAY).from(14).to(19))
  .with(commitment().plannedFor(days(5)).completedIn(days(1))
  .with(commitment().plannedFor(days(1)).completedIn(days(3));

But I could also define a specification for the test data that is more concise

createUser()
 .with(overtime().of(hours(13).including(weekends()))
 .with(commitments().completedBeforeDeadline(0.5))

and is used by the test framework to generate actual data for the tests.

Having this kind of abstraction in the test (only a few most important variables), I am immediately able to grasp the idea of test. Also, I can write much more combinations of test in a short time.

I know we haven't invented a wheel, so I would like to read more about this approach, its risks etc but don't know how to search for it. How do you call it? Is it somehow related to BDD? Or what?

share|improve this question
    
I think this is called "the builder pattern". Regardless of its name, it sounds like a good idea. –  user246 Sep 12 '13 at 22:23
    
Builder pattern just creates a (usually immutable) object via fluent interface. Here I do something more, see my comments to Dale's answer. –  dzieciou Sep 13 '13 at 6:07
add comment

2 Answers 2

That is the Builder pattern.

It is not specific to BDD. It's fairly common in integration tests and functional tests. It's less used in unit test, which typically have less need for complex data setup.

A test data builder might either construct data, or find relevant data in a database. (I strongly prefer building data from scratch, but that's a rant for another day.)

Your second example focuses very nicely on the intention of the data.

If the essence of the test is that the overtime be on weekends, the first example clutters the test with incidental details that do not clearly express the intention. A reader could maybe puzzle it out, especially if the test name indicates that the test has something to do with weekend overtime.

share|improve this answer
    
+1 RE focusing on the intention of the data. The builder pattern lets the system fault in the parts of the test data that don't matter to the test. I used this with great success in a previous job. –  user246 Sep 12 '13 at 22:39
    
I didn't mean the builder pattern. Builder pattern just creates an object with fluent interface, and often makes it immutable. I meant moving from actual data definition (1st ex) to data specification (intention as you call it, 2nd ex). Data specification to be designed maybe according to Specification pattern, as described by Martin Fowler. There should be also something that will generate sample definition from specification. I would call this translating part SpecificationResolver, but maybe there is already a better name in design patterns. –  dzieciou Sep 13 '13 at 6:03
    
@user246, what did you mean by "lets the system fault"? Didn't you mean "fake", not "fault"? –  dzieciou Sep 13 '13 at 6:05
1  
Ah, I see I misunderstood the thrust of the question. I don't have a name for the shift you made. I just keep applying two principles: 1. Name every important idea. 2. Hide incidental details. Easier said than done, given the slippery words important and incidental, but you seem to have an unusually good grasp of those ideas. –  Dale Emery Sep 13 '13 at 9:27
1  
@dzieciou I may have misunderstood the question as well. By "fault in" I mean providing defaults for the necessary data that I don't specify. –  user246 Sep 13 '13 at 12:02
add comment
up vote 2 down vote accepted

That is the Specification design pattern.

When I was writing my question (I'm the OP) I did not know Eric Evans and Martin Fowler already defined it. It originally addressed the problem of validation whether a given candidate object matches certain criteria (a specification), by defining Speficiation interface with a isSatisfiedBy(Candidate c) boolean method. Specification was said to be also a solution to the problem described in the OP question: to generate test data matching certain specification. The authors generalized it to:

You need to describe what an object might do, without explaining the details of how the object does it, but in such a way that a candidate might be built to fulfill the requirement.

I think their example of this problem, even if not discussed in testing context, is more illustrative than mine (in the original question):

The customer provides a route specification for the shipment. The shipping company will use the specification as a constraint in coming up with a route.

Unfortunately, you won't find much details how to implement that in their article. I would start from a generic interface CandidateGenerator with the generateCandidateMatching(Specification s): Candidate method. The interface can be implemented using one of the following patterns:

  • Factory pattern, that constructs a candidate matching the specification, as user246 suggested.
  • Finder "pattern", that searches for a candidate (e.g., in a database), e.g., RouteFinder would query route database for a route, by translating a specification into one or more database queries:

    enter image description here

  • Factory with Finders, a combination of above: search database for available itineraries and then construct a new route with those itineraries.

Design guidelines

  • Having specification implemented you can always verify whether CandidateFinder or CandidateFactory returns correct candidate by passing it to the isSatisfiedBy() method.
  • Specification can be a composition of smaller specifications (as Evans and Fowler envisioned), so generating a final candidate can be done by generating candidates for smaller specification and then combining them together.
  • Naive implementation of a Finder would be to iterate over all candidates and evaluate specification over each of them. It is naive, because this approach might be inefficient and not always possible (you might not have access to all candidates); that's why DBMS build query plan for a query.
  • Specification can be defined with the Builder pattern, as Dale Emery proposed.
share|improve this answer
1  
There are two distinct ideas here: a specification and a finder. The latter looks for objects that match the former. Sometimes it is also useful to couple a specification with a factory. This can be especially useful for constructing test data. –  user246 Sep 14 '13 at 19:52
    
Good point but I'm not sure if I see a distinction between factory and finder. We have a DB of possible itineraries and the SUT relies on those data. My intuition is that RouteFinder will always create route with itineraries present in DB, while RouteFactory.createRouteFor(specification) could be useful to create also routes with itineraries not available in DB. Shortly, the factory is not so coupled with the SUT as finder. –  dzieciou Sep 14 '13 at 20:12
    
@user246: But on the other side, RouteFinder constructs a new route just like factory does. It does not find existing route in DB, it just uses existing itineraries to create a new route. –  dzieciou Sep 14 '13 at 20:14
1  
I may have used a confusing combination of terms. You could imagine using a specification to search for matches among existing data (e.g. reading from a database), or to construct new data (e.g. write to a database). Sometimes it may be easier to construct what you want than to try to search for it, especially as the number of criteria increases. Other times, you may really want to use existing data. –  user246 Sep 14 '13 at 20:28
    
@user246, Thank you. I included your comments in my answer. –  dzieciou Sep 15 '13 at 7:36
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.