Software Quality Assurance & Testing Stack Exchange is a question and answer site for software quality control experts, automation engineers, and software testers. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Our company is being asked to show that on a regular basis we reconfirm the ability of our software to perform certain tasks. The section in question from the ISO 9001 standard is as follows:

7.6 Control of Monitoring and Measuring Equipment

"When used in the monitoring and measurement of specified requirements, the ability of computer software to satisfy the intended application shall be confirmed. This shall be undertaken prior to initial use and reconfirmed as necessary."

The bolded portion is the part we're having problems with. We have a slew of in-house testing software (Where there are two main programs, and about 30 smaller programs), that runs hundreds if not thousands of different scripts for hundreds of different products every single day. They've recommended that we regularly place previously failed and stocked product on our test stations to basically prove that at any given point in time a "pass" is still a "pass" and a "fail" is still a "fail." There's no way we can get the coverage to prove every single failure is working as intended. We've been brainstorming ideas, including using mock product, but all are seemingly extensive and rather useless on our end. What's a good way to do this?

share|improve this question

closed as off-topic by TESTasy, bish, Kate Paulk, Bruce McLeod Jan 15 at 4:47

This question appears to be off-topic. The users who voted to close gave this specific reason:

  • "Questions about software engineering unrelated to testing are off-topic here, but can be asked on Programmers Stack Exchange." – TESTasy, bish, Kate Paulk, Bruce McLeod
If this question can be reworded to fit the rules in the help center, please edit the question.

Welcome to SQA, MGSoto. Who is "they": management within your company or an outside ISO 9001 certification organization? Is your testing software really supposed to correctly categorize every single unit, or are you just shooting for a certain margin of error? – user246 Jun 1 '12 at 17:01
Thanks @user246! "They" is an external auditor. I'm not really sure I understand your second question, are you asking in terms of auditability or testing in general? – MGSoto Jun 4 '12 at 15:48
I apologize - I misunderstood your original question. My second question is not applicable to what you asked. – user246 Jun 4 '12 at 15:56
That sounds like a pretty crazy level of paranoia on the part of your auditor. The automated (re-)testing itself should be enough to satisfy that part of the standard. It's the product that needs reconfirming, not the tests that confirm the product. Otherwise, how far do you go? Tests for the tests? Tests for the tests for the tests? Where do you stop? – Mal Ross Jun 7 '12 at 12:06
As an addition to the comment from @MalRoss, would the auditor be satisfied with running a set of benchmark scripts i.e. you know the outcome of each script after interrogating the data, therefore proving that a test that should fail in your software will fail and one that should pass will pass? – SheyMouse Jun 21 '12 at 11:35

Browse other questions tagged or ask your own question.