Should you write tests that are basically for no-op to catch errors? (e.g. test a transition does NOT happen in a state machine)

Question

When writing tests for a state machine, should you also test conditions that would not trigger transitions to make sure they don't?

E.g. for a software requirement

transition from A to B if x is between 5 and 12

would you also write a test for x = 3 to make sure that the transition does not happen and the state machine stays put?

In the source code x = 3 would be a no-op and nothing would happen in that case. But for a software tester just working off the software requirement and who does not know, e.g. if values in the conditions in the source code are correct, I would argue that it is legit to write tests for that.

What are you thoughts on this? Are there best practices or paradigms to help with this case?

Answer 1

The short answer is Yes! You should write a test for conditions that don't transition the state machine.

In the above case, x = 3, you are utilizing the principles of Boundary Value Analysis and Equivalence Classes. These are excellent ways to create test cases for the right conditions, both positive and negative.

The next question would be, what do I assert if my test case is: "when x = 3, I do not transition". If you are keeping track of the state, assert transition = false. Or, if you know what the next states are, you can assert that nextState1 = false. Ultimately, you are asserting true if you transition and false if you stay put.

These are important areas to test because in the future, what if you change the states/transitions, e.g., you add more states/transitions, or you remove states/transitions? Or, maybe you decide to change the transition criteria to be transition from A to B if x is between 2 and 15. Don't you want to catch possible future changes to prevent regressions?

Answer 2

If you were to apply equivalence partitioning, you would end up with a few partitions:

• x < 5
• 5 <= x <= 12
• x > 12

Given that requirement, I would expect at least those three test cases to assert behavior.

You can also apply boundary-value analysis. The values x = 5 and x = 12 are the boundaries of when something interesting happens. It may be useful to explicitly add those tests and ensure your test for the action occurring is some other value in the range 5 < x < 12.

Depending on what the full range for x is, you may also want to consider interesting cases, like x < 0, x = 0, and x at the largest and smallest values.

The full range of test cases and level of rigor depends on the system. For life-critical systems, I would expect that most of the cases I identify would be covered. For the least critical systems, you could get away with as little as two or three test cases based on the equivalence partitions.