Where I work we use an offshore company to augment several internal departments. I work in the QA team, and concerns have been aired about the level of confidence we can have in the test work we have been assigning to workers at this company.
I have suggested that it may be a good idea to use error seeding to get a metric of our confidence level in code that has been tested by this company, and compare it with the same metric as used internally.
This is a technique I learned about on an ISEB testing course, however I have not met many people who have used it in a real-world environment, and so would like some input as to what best practices may be. The following are things which I have thought of, but obviously it would be better to learn from the experience of someone who has tried this before.
- Have developers seed multiple (n) bugs per testable item
- Have developers track these bugs in a system that QA is not able to access
- Alert the tester for said testable item to how large n is (approx? absolute?)
- For each bug that tester raises, alert them as to whether the bug was seeded or not
My reasoning is that if we make this metric visible and transparent, it will also give testers feedback as to how rigorous their testing is.
I would welcome input as to any practical issues that may make this difficult or even unworkable. If this is implemented I shall leave an answer updating everyone on my own experience with this.