Take the 2-minute tour ×
Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. It's 100% free, no registration required.

Defect Seeding seems to be one of the few ways a development organization can tell how thorough an independent testing group is. I'm a fan of using metrics to help counter overconfidence biases, and drive discussions around facts. With that said, I haven't seen Seeding Defects used in practice.

Are there best practices above and beyond what McConnell explained? Are there public examples where this has been done? In the absence of the above, any thoughts on why it hasn't been done more?

Thanks in advance!

share|improve this question

migrated from programmers.stackexchange.com Jul 6 '13 at 22:28

This question came from our site for professional programmers interested in conceptual questions about software development.

2  
I've never used defect seeding, but I suspect the problem is one that McConnell pointed out: you may forget to remove you seeded defects. The places that are best served by using defect seeding to determine the effectiveness of their testing processes are also the places that would be most impacted by leaving a defect in the system. I can think of ways to mitigate this problem, but it still comes down to a question of time and budget to deal with the seeding and testing not only the seeded code, but the unseeded/releasable code. I would like to here from someone who has tried it, though. –  Thomas Owens Jun 24 '13 at 14:04
2  
@ThomasOwens: Create a branch or diff in your source control system that only includes the seeded defects. –  Robert Harvey Jun 24 '13 at 16:57

2 Answers 2

I, too, have never seen this in practice. In my experience, QA departments are always short of resources and/or time. So taking extra time to validate the tests instead of the software they are meant to test would never gain traction, in my experience.

What I have seen is QA groups know the coverage of their tests. Where the coverage is thin, they build them up. Typically, this starts with manual tests, where the test scripts are documented in Word documents (for example). If the script doesn't match the software, then either the software is at fault (a bug) or the script is updated (assuming the feature has been modified).

Where tests can be automated, then this is done (although not by all QA departments I've seen). Having the automated tests allows them to test more faster. the tests do require maintenance, but it's time worth spending (in my opinion).

share|improve this answer

To create a good test, the defects must be truly random, and randomly distributed. If we simply make random changes to source code, most of those changes would result in code that does not compile, and if it compiles, a percentage of those defects are trivial to find, especially if the software contains a lot of pre-condition and post-condition checks. So you need good tools such as Javalanche or NinjaTurtles. At the Javalanche site, you will find studies using mutation testing.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.