xUnit tests patterns for testing threaded applications

Question

Are there either any xUnit (or otherwise) design patterns or anti-patterns for testing threaded code?

Answer 1

Have you seen a http://xunitpatterns.com? A lot of great material there.

In my experience dealing with threaded behavior in tests just makes things needlessly complicated and not worth the effort. Try to isolate tasks from each other by using mocks/stubs as much as you can.

There is one particular pattern however that helped me a lot: design by contract. Which I implemented by inserting assertions in the application code and stress testing the system in attempt to trigger their violations.

YMMV of course, but for my projects this approach (isolation and stress testing) proved to be more effective than trying to write tests for specific use cases.

Answer 2

The most useful test pattern for testing threaded applications is The Exigent Avoidance pattern. It is defined is as follows:

Don't.

Most people bake multi-threading into the application code that they write. This is malpractice.

^{(And since the severity of a malpractice is a function of its inherent malignance, which is already pretty bad, multiplied by its pervasiveness, which is almost universal, it is a very severe malpractice. The severity of this malpractice is of truly astronomical proportions.)}

The bane of multi-threading is race conditions. Code which has been written with specific knowledge of multi-threading utilizes synchronization (locks) to avoid race conditions; however, a little fact which seems to go unnoticed by most people out there is this:

There exists no unit test that will ever detect a race condition.

This has an inescapable and grave implication:

All multi-threaded code is untestable.

Testing is not something that you can apply as an after-thought; you cannot design and build a system and only then start to worry how you are going to test it; testing is so important, that you have to begin with architectural choices that facilitate it.

In order to build software systems that utilize multi-threading and are also testable, you have to start treating multi-threading not as an application-logic concern, but instead as a system deployment and wiring concern. The only code that should know anything about threading should be the relatively small amount of initialization code that runs during system startup and wires up the system before proceeding to show-time.

All application-logic must be completely agnostic of the threading regime under which it has been wired to run: it should run just the same single-threaded, or in multiple discrete threads, or in a thread pool, or in a fiber pool. The wiring code should be able to choose a different threading regime, and your application-logic code should continue working the same way, without having the slightest clue that this happened. (The only difference should be performance.)

This means that application-logic components have to be decoupled so strongly, that there can be no possibility of race conditions when they are run in parallel.

The way we usually achieve this is with event-driven systems in which components share data not by means of synchronization, but instead by passing immutable messages via message queues. Essentially, the only piece of code in the entire system which utilizes synchronization is the queue abstract data type that is used to implement the message queues. The idea is that we should be able to get at least that little part right.

Once you go through the pains necessary to achieve all of this, testing the application-logic becomes quite simple and straightforward: there is no need for any special testing patterns for testing multi-threaded code, because when you test components, you wire them to run under a strictly single-threaded regime.

Incidentally, this also makes debugging magnificently easier.