Are each of these considered Best practice:
- Storing test cases in comments : Absolutely not, it makes automated tests far too difficult.
- Storing test cases in the same file as the the source code : Not really, it's too distracting.
- Storing test cases in the same repository as the source code : Yes, especially if developers do TDD.
- Storing test cases in a separate test repository : Yes, but only if required by procedure.
Firstly, the idea of storing test cases in comments in the code seems like the worst of all worlds. You can't easily use a testing framework to automate your tests, you have to do some magic to extract the tests and run them (so you get no help from your IDE when writing them) and your code is bloated with tests which obscures the purpose of the code.
Storing test cases in the same source file as the production code under test is the simplest option. But without a lot of pre-processor directives or annotations you will end up with your test code bloating your production code unnecessarily, and depending on how you have structured your code, you may end up accidentally exposing internal implementation to users of that code.
The best option is to storing test cases separately from code and structure your code to support that. This has a number of advantages, as described in other answers, but there are also pitfalls. Luckily there are often ways to get around them, which others do not seem to have discussed.
For instance, if you want to unit test some private behaviour of a class, depending on the language/environment, you may have three options:
- Put the tests in the class you want to test.
- Put the tests in another class/source file & expose the private methods you want to test as public methods (yeuch).
- Use a testing environment that allows you you keep test and production code separate, yet still allow testing code access to private methods of the production code.
In the Eclipse world, 3. can be achieved by using fragments. In the C# world, we might use partial classes. Other languages/environments often have similar functionality, you just need to find it.
Blindly assuming 1. or 2. are the only options would be likely to result in production software bloated with test code or nasty class interfaces that wash their dirty linen in public. *8')
I find it difficult to imagine doing anything other than storing test cases as code under revision control, just like production code. You might prefer test code to be in a different repository, possibly even with different access permissions for test programmers and production programmers (if you separate the two), but it just as important to know which version of the tests you are running as it is to know which version of the code is being tested.