Why test code should not be included in software application?

Question

We are developing Android application and the developers feel it is difficult for us to write integration/unit tests. I read majority in Test cases in source code saying that test code should not be included in software application. However, the developers think it is easier to write unit/integration tests inside the application. He could input some magic number in non-production version of the application and the application starts to perform unit/integration tests, not just debug code that only set-up preconditions.

I know such test would increase size of application. It is disabled in production version so it is safe from being used by user but it also means we cannot test our production version. Is there any other reason that we should not write unit/integration tests within our application but write tests that run independently in test instrument/test runner?

Answer 1

The approach I've seen most often is to have the unit tests as a separate project in the application solution, so they can be run against production code as part of the build process, but are not included in the production code.

Some of the reasons for the separation are:

• Logical separation. You want your unit tests to catch issues with initialization, global variables, and scoping, which is more difficult if they're part of the main code base (depending, of course, on the language you're using).
• No need to worry about compile-time flags - it's remarkably easy to miss adding a compile-time flag to keep your test code out of your production build. If you have a separate testing project in the solution, this isn't an issue - but you also get to keep the convenience of matched check ins, because it's all in the version control system, right?
• No shortcuts. Because the test codebase is in a different project, you have to perform all the initialization, setup, and teardown. You can shortcut if the test code is part of the application project - and those shortcuts can mean invalid tests.
• You don't have to link test data to the application code. This can be a big issue for large and/or complex applications. The test data source is linked to the test project, not the application code, so it doesn't slow the build and check in/check out process for the application code (and you are running the unit tests for every full build, right?)

Answer 2

If the test code is excluded by a compiler option for the product version I see indeed no reason why this should be a questionable behaviour.

On the contrary this is sometimes the only way to do unit tests, e.g. if you want to test static functions in C. There is noc other way to call them directly but from inside the translation unit.

Secondly you can always be sure the revisions of the test cases and code match.