Computer scientists are actively researching model-driven testing as a means of testing applications automatically. The papers I've read are interesting, but so far the modelling technology appears too immature to apply to anything beyond "toy" applications.
Have any of you used model-driven testing in a professional capacity (i.e. beyond a school project), and if so, what were your experiences?
Yes - when it's applicable. Model-based testing is just another test design technique - but one that works quite well when the application under test (or feature area under test) behaves like a finite state machine.
At Microsoft, we've used model-based testing extensively for features that behave like a state machine. Protocol testing is probably the largest example (protocols like TCP/IP or SIP) are great examples of state machines and can be tested well using MBT (we use Spec Explorer to generate the models and tests.
Some other success stories on model-based testing with Spec Explorer are in this discussion.
I believe model-based testing can be good not only for generating test-cases from model but also for model checking. Checking the correctness of abstract model can be wise when testing the implementation completely is too expensive (hard setup, expensive prototyping).
Real example
In our system users can work concurrently but cannot perform tasks that write/read to the shared entities. When such a situation happens UI should deactivate the button for the blocked task. We have many shared entities and many tasks that use them, so assigning the right combination of entities that will be locked by the task and foreseeing which task will be blocked by which task is pretty hard.
To verify the assignments, I wrote a simple tool that prints which task blocks which task based on the given matrix of entities shared by tasks. There was no formal checker, whether the matrix is correct. I was the oracle to decide whether the locks are not too much/too little aggressive.
Lessons learned
However, the model checking helped us find sub-optimal configuration of locks.
The model checking does not replace testing of the real system. Some locks were found later implemented incorrectly.
The model checking helped us identify a number of critical test cases for testing the real system.
Personally, model based testing completely does my head in!
I like to think that I am not the dumbest person out there, but even after sitting down with HWTSAM, rolling up my sleeves and trying to get into it, I simply couldn't get it to work for the web based application I was testing at the time I tried.
That said, I don't think that means that it can't work, I think that means that effective Model based testing has a steep learning curve before you can get any results and real world benefits over and above other techniques.
I also think that in "real world" testing where time is tight and people don't have that much time to spend learning model based testing (using Spec Explorer in Visual Studio), at the moment is simply too much time invested for not enough results.
Something like Specflow and writing tests myself without the model gives me much more bang for my buck when it comes to pounding out quick test cases.
