Documenting Unit Tests

Question

We follow a BDD approach to development, we have our testers creating scenarios from a specification, and these are then given to our developers before any work is started. However, we find that some of our developers often choose to write extra unit tests to help then write the code required pass the scenario.

The problem is that only some of our developers do this, we don't discourage this in anyway at all, however, we think that it would be unfair to ask another developer that doesn't do this to maintain these tests if he made a change to that code. Remember we have a large collection of scenarios that are executed to ensure that the application behaves correctly.

Should we be documenting these unit test somehow, the problem is that our scenarios don't assume any implementation details and the extra unit tests are often to help with testing some of these details.

Answer 1

I read two questions: (1) should we document the tester-written unit tests and (2) how should we treat tests that assume implementation details?

It's important to distinguish between scaffolding and unit tests. (Please don't get caught up in my terminology; I'm just trying to make a point about some concepts.) Scaffolding is code a developer writes as part of creating a feature. They take whatever form the developer wants them to take. They might be tests against interfaces. They might be tests against internals to the implementation that are not in any way a part of any specification or requirement. The may be tests that produce a pass/fail result, or they might emit blobs of data that only made sense to the developer while he was coding and debugging.

Once the code is working, the scaffolding is probably just a pile of junk. Even if the interfaces don't change, if the scaffolding assumes implementation details, there is no guarantee or expectation that the scaffolding will do anything useful.

No one should have to maintain someone else's scaffolding, and it should be the developer's job to document their scaffolding if they choose to keep it around.

Unit tests (for the purposes of this answer) are tests against an interface that is a response to a requirement/specification. Unlike scaffolding, unit tests restrict themselves to public interfaces. Unit tests are written to produce a pass/fail result. Most importantly, unit tests should still pass if the interface remains the same but the implementation changes.

I think unit tests should have some kind of accompanying documentation. Perhaps it's comments in the code, or perhaps it's an external document, although you need to remember that as the tests change, someone will have to update the external document too.

Answer 2

Your testers deliver the tests scenarios that should be implemented by the developers and some of your developers write more tests than required. Give them a raise!

I recommend to review the extra tests and check if they make sense. Encourage all developers to write more tests to put their knowledge into tests. They know implementation details, so they can write more valuable tests. They have to make sure that their code works, so they should have the freedom to write tests to prove/tests their own work.

Document these tests but preferably only in the code. If the tests fail with a code change, who changed the code has to fix the tests, adapt the test, or remove it in case it is not longer required.

See this as two different kind of tests: Testers specify integration style tests looking from the outside using specs, developers look from the inside using implementation details to prove their work.

Both tests are important and you should make both things mandatory.

Hope that helps a little and by the way, we have a motto here: "You touch it, you own it." So whatever tests are there, they have to be fixed.

Answer 3

In the traditional sense of the word a unit test is designed to test the smalled functional unit of software such as s single method or function. Unit tests are intended to provide limited testing of the functional capabilities of a function or method in isolation.

Typically, unit tests are written and maintained by developers because they should be used by the developers to qualify their work before checking in their code. Unit tests are especially helpful when refactoring code, for initial validation after code churn, and as pre-check-in suites by the dev team in a continuous integration enviornment.

So, would it be unfair to ask developers to maintain unit tests (even if they are written by other developers). To paraphrase Hunt & Thomas, 'developers are paid to write working code.' (Pragmatic Unit Testing In C# with NUnit) And unit tests can a bit of sanity that the unit of code at least does what the developer thinks it's supposed to do. If their code fails a unit test their change will likely break the build. If their code causes the unit test to become outdated, there is likely a change in underlying architecture or functionality and as a tester I would want to know that.

Bottom line units are quite different tools as compared to the behavioral tests derived from a BDD approach and have different purposes.

Answer 4

Some of your developpers are probably doing a combined BDD/TDD approach as described in MSDN Magazine: BDD Primer - Behavior-Driven Development with SpecFlow and WatiN.

I see unittests as a kind of executable documentation that fail if the documentation (=testcode) or the some productioncode (that is called by the testcode) is wrong. Do you consider "Updating Softwaredocumatation as unfair if it is outdated"?

As long as Unittestmethods have good names and are easy to read i donot think that there is much need for extra ducumentation.

I totally agree with ReneS answer (+1)