PHP: code instrumentation library to capture/record new unit test cases?

Question

I have a library that seems to work as intended. I would like to add some instrumentation to the library to capture (some) current input and the result and save that as a test case. Is there a library or tool for PHP to do this?

In short: given (1) current source code and (2) some example execution runs, output (3) test cases to verify that these execution runs repeat the same after the source code has been modified without repeating the example execution runs manually.

Ideally, I could mark a class or a function or method as "experimental" and this tool or library would generate test cases for this class/function/method that submits recorded input and saves computed output. Generating test cases for all methods executed during the example execution run would probably generate too much noise.

Obviously tests generated this way would be far from perfect but I'd guess these tests would be good at capturing accidental code regressions.

Can you suggest anything? I'm aware of Selenium but recording tools for that seem to test much bigger parts of the system. I'm looking more for a tool that is closer to unit testing and possibly would generate test cases (in source code form) that are understandable enough to be improved in the future.

Answer 1

This sounds like you might need to build some custom code, I haven't heard of anything that does this off the shelf, for any language. However, here's an approach I think might work;

Assuming your using well structured objects, create a facade object that wraps the behaviour of the class under test, and present the facade to your normal application. Implement the facade such that it records log output to file which can be reconsituted into runnable methods.

Two obvious ways to handle the logging; dump a unique log file per instance of your facade, and construct the log messages such that they 'write code'

Public doAction($a){
  // start by telling our "test" to call the method
  //serialise the input so it can get logged properly
  $serialInput = serialise($a);
  Log.info("$result = $instance->doAction(deserialise($serialInput));");
  // now really call the method
  $realResult = $realImplementation->doAction($a);
  //serialise the returned object so we can persist it in the log
  // and do some assertions to check things
  $d = serialise($realResult)
  Log.info("assertEquals($result, deserialise(".$d.");");
  // really return the method to keep our facade intact
  Return $realResult;
 }

You'd need to construct new instances and functions when constructors are called, and will probably end without closing tags, but that's easy to add.

It's a fragile approach, but with some clever reflection could infact have a general purpose implementation built.