Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Computer scientists are actively researching model-driven testing as a means of testing applications automatically. The papers I've read are interesting, but so far the modelling technology appears too immature to apply to anything beyond "toy" applications.

Have any of you used model-driven testing in a professional capacity (i.e. beyond a school project), and if so, what were your experiences?

share|improve this question
add comment

3 Answers

up vote 8 down vote accepted

Yes - when it's applicable. Model-based testing is just another test design technique - but one that works quite well when the application under test (or feature area under test) behaves like a finite state machine.

At Microsoft, we've used model-based testing extensively for features that behave like a state machine. Protocol testing is probably the largest example (protocols like TCP/IP or SIP) are great examples of state machines and can be tested well using MBT (we use Spec Explorer to generate the models and tests.

Some other success stories on model-based testing with Spec Explorer are in this discussion.

share|improve this answer
1  
"Yes - when it's applicable." Yes! –  Joe Strazzere May 31 '11 at 12:47
    
Who writes the test models ? the same team that codes the feature or the test team ? –  Rsf Jun 25 '12 at 13:34
    
Ususally (actually, in most cases), the test team writes the models. –  Alan Jun 27 '12 at 0:58
add comment

I believe model-based testing can be good not only for generating test-cases from model but also for model checking. Checking the correctness of abstract model can be wise when testing the implementation completely is too expensive (hard setup, expensive prototyping).

Real example

In our system users can work concurrently but cannot perform tasks that write/read to the shared entities. When such a situation happens UI should deactivate the button for the blocked task. We have many shared entities and many tasks that use them, so assigning the right combination of entities that will be locked by the task and foreseeing which task will be blocked by which task is pretty hard.

To verify the assignments, I wrote a simple tool that prints which task blocks which task based on the given matrix of entities shared by tasks. There was no formal checker, whether the matrix is correct. I was the oracle to decide whether the locks are not too much/too little aggressive.

Lessons learned

  1. I did this model checking definitely too late, as we already have the system implemented, and we discovered the whole idea of locking was somehow flawed, because it did not give us the right level of lock control, i.e., there might be no configuration (matrix) that would satisfy that requirements completely.
  2. However, the model checking helped us find sub-optimal configuration of locks.

  3. The model checking does not replace testing of the real system. Some locks were found later implemented incorrectly.

  4. The model checking helped us identify a number of critical test cases for testing the real system.

share|improve this answer
add comment

Personally, model based testing completely does my head in!

I like to think that I am not the dumbest person out there, but even after sitting down with HWTSAM, rolling up my sleeves and trying to get into it, I simply couldn't get it to work for the web based application I was testing at the time I tried.

That said, I don't think that means that it can't work, I think that means that effective Model based testing has a steep learning curve before you can get any results and real world benefits over and above other techniques.

I also think that in "real world" testing where time is tight and people don't have that much time to spend learning model based testing (using Spec Explorer in Visual Studio), at the moment is simply too much time invested for not enough results.

Something like Specflow and writing tests myself without the model gives me much more bang for my buck when it comes to pounding out quick test cases.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.