What coverage measures/metrics are useful for unit testing and for integration testing of Java or other object-oriented systems?
We would like to establish a better test process. Our software systems are (mostly) developed and maintained by external providers (including implementation of change requests). As of now, they do the development and (hopefully) some testing and provide us with the resulting software. Then we do the system testing.
Now we would like to make some specifications of measures which have to be fulfilled by the provider in order for us to accept the software and start system testing.
There are some techniques like statement testing and decision/branch testing which for which some tools exist (e.g. Cobertura, EclEmma), and which seem to be relatively easy to use. These could of course be somewhat helpful.
Unfortunately they seem to be insufficient for object oriented systems, since the complexity of object-oriented systems mostly is due to the relations between classes and objects, not due to complex control flows. Required statement coverage (e.g. 100%) or branch coverage could be achieved relatively easy. Nevertheless, there could be many quality problems which cannot be identified by these tests.
Are there other techniques and corresponding coverage measures which are more adequate? If yes, can you recommend tools?
After thinking about it, and without having found better measures/metrics for object-oriented systems, I contemplate the following approach:
- Require statement coverage of 100%, maybe with justified exceptions of some classes or (groups of) methods.
- Maybe require decision/branch coverage of 100% for all classes/methods which are not excluded (see previous list item), depending on the project.
- Additionally to these measures/metrics, and prior to development, perform design reviews to get confidence in the approach the provider chose.