Defect Seeding seems to be one of the few ways a development organization can tell how thorough an independent testing group is. I'm a fan of using metrics to help counter overconfidence biases, and drive discussions around facts. With that said, I haven't seen Seeding Defects used in practice.

Are there best practices above and beyond what McConnell explained? Are there public examples where this has been done? In the absence of the above, any thoughts on why it hasn't been done more?

  • 2
    I've never used defect seeding, but I suspect the problem is one that McConnell pointed out: you may forget to remove you seeded defects. The places that are best served by using defect seeding to determine the effectiveness of their testing processes are also the places that would be most impacted by leaving a defect in the system. I can think of ways to mitigate this problem, but it still comes down to a question of time and budget to deal with the seeding and testing not only the seeded code, but the unseeded/releasable code. I would like to here from someone who has tried it, though. Jun 24, 2013 at 14:04
  • 2
    @ThomasOwens: Create a branch or diff in your source control system that only includes the seeded defects.
    – Robert Harvey
    Jun 24, 2013 at 16:57

2 Answers 2


I, too, have never seen this in practice. In my experience, QA departments are always short of resources and/or time. So taking extra time to validate the tests instead of the software they are meant to test would never gain traction, in my experience.

What I have seen is QA groups know the coverage of their tests. Where the coverage is thin, they build them up. Typically, this starts with manual tests, where the test scripts are documented in Word documents (for example). If the script doesn't match the software, then either the software is at fault (a bug) or the script is updated (assuming the feature has been modified).

Where tests can be automated, then this is done (although not by all QA departments I've seen). Having the automated tests allows them to test more faster. the tests do require maintenance, but it's time worth spending (in my opinion).


To create a good test, the defects must be truly random, and randomly distributed. If we simply make random changes to source code, most of those changes would result in code that does not compile, and if it compiles, a percentage of those defects are trivial to find, especially if the software contains a lot of pre-condition and post-condition checks. So you need good tools such as Javalanche or NinjaTurtles. At the Javalanche site, you will find studies using mutation testing.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.