Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

I have a library that seems to work as intended. I would like to add some instrumentation to the library to capture (some) current input and the result and save that as a test case. Is there a library or tool for PHP to do this?

In short: given (1) current source code and (2) some example execution runs, output (3) test cases to verify that these execution runs repeat the same after the source code has been modified without repeating the example execution runs manually.

Ideally, I could mark a class or a function or method as "experimental" and this tool or library would generate test cases for this class/function/method that submits recorded input and saves computed output. Generating test cases for all methods executed during the example execution run would probably generate too much noise.

Obviously tests generated this way would be far from perfect but I'd guess these tests would be good at capturing accidental code regressions.

Can you suggest anything? I'm aware of Selenium but recording tools for that seem to test much bigger parts of the system. I'm looking more for a tool that is closer to unit testing and possibly would generate test cases (in source code form) that are understandable enough to be improved in the future.

share|improve this question
    
What would you consider to be the computed output of an experimental function: just its return value, or also any side-effects? I assume the latter; otherwise, you cannot guarantee that the experimental function behaves exactly as before. –  user246 Feb 7 '12 at 3:27
    
I would be happy with a tool that just records the return value for a collection of input values (logically testing pure functions). I don't believe there can be a generic tool that can capture all side-effects (e.g. a piece of algorithm may call to SSL protected remote sites, read from /dev/random and write to a pipe and/or file). I think that only a human can write tests that check the side-effects. –  Mikko Rantalainen Feb 7 '12 at 9:27
    
Perhaps it would be better to migrate this to StackOverflow. –  user246 Feb 24 '12 at 2:00
add comment

1 Answer 1

This sounds like you might need to build some custom code, I haven't heard of anything that does this off the shelf, for any language. However, here's an approach I think might work;

Assuming your using well structured objects, create a facade object that wraps the behaviour of the class under test, and present the facade to your normal application. Implement the facade such that it records log output to file which can be reconsituted into runnable methods.

Two obvious ways to handle the logging; dump a unique log file per instance of your facade, and construct the log messages such that they 'write code'

Public doAction($a){
  // start by telling our "test" to call the method
  //serialise the input so it can get logged properly
  $serialInput = serialise($a);
  Log.info("$result = $instance->doAction(deserialise($serialInput));");
  // now really call the method
  $realResult = $realImplementation->doAction($a);
  //serialise the returned object so we can persist it in the log
  // and do some assertions to check things
  $d = serialise($realResult)
  Log.info("assertEquals($result, deserialise(".$d.");");
  // really return the method to keep our facade intact
  Return $realResult;
 }

You'd need to construct new instances and functions when constructors are called, and will probably end without closing tags, but that's easy to add.

It's a fragile approach, but with some clever reflection could infact have a general purpose implementation built.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.