2

Working on my master thesis, today I found this site (I previously asked on programmers) so I hope you will not mind if I ask here to get more great answers. The thing is that I have to classify testing into validation or verification but the standards differ greatly:

  1. ISO/IEC 12207: All testing is validation (unit, integration..just all)
  2. CMMI: Verification methods are testing, analysis. The same for validation.
  3. IEEE 1012: Validation and verification reffered as V&V for all activites, thus no discinction.

IEEE states just this:

The validation process provides supporting evidence that the software satisfies system requirements allocated to software, and solves the right problem (e.g., correctly models physical laws, or implements system business rules).

which is IMHO absolutely different than ISO, which lists testing as activity to make sure that requirements for intended use are fulfilled.

CMMi verification: load, stress, functional...testing (exactly what ISO mentioned to be validation)

When asking the famous questions, "Do I build it right" and "Did I build the right thing", it is not helpful from testing point of view.
E.g. How can unit testing or testing specific requirement be a validation? (according to ISO) I would say I am verifying particular requirement against specification, yet ISO consider all testing to be a validation. I emphasize that this is about theory, I know that in practice we do not care much whether its validation or verification, yet I would need to present some unified view.

Improve this question
3
  • Can you correct the next to last sentence, the one that begins with "How unit testing"? I think I understood your question up until that point.
    – user246
    Commented Nov 2, 2012 at 16:51
  • Is that the right IEEE standard, as IEEE 1202 is flame testing of cables for use in cable tray. I thought IEEE 829 was the software testing one?
    – Andrew
    Commented Nov 2, 2012 at 18:42
  • 1
    @Andrew Sorry, it is IEEE Std 1012 Standard for Verification and Validation.
    – John V
    Commented Nov 2, 2012 at 19:37

6 Answers 6

Reset to default
2

At risk of rehashing previous threads, verification (according to ISO12207) is ensuring the correct processes and standards are followed and that all requirements are traceable though the lifecycle, not the testing of actual requirements. Testing is a validation activity - you are validating the code against the requirement.

At the end of the day, your customer (via the contract or their specification) or your in house standards will define which you follow.

Or in the case of your thesis, YOU decide, and justify it.

This is the great thing about Standards - there are so many to choose from!

As an aside, the IEEE have fully adopted ISO/IEC 12207, and they are harmonising their standards with that Process Model (as are BSI in the UK) so I would expect the next iteration of IEEE 1012 to be more discerning.

With regarding the line between V and V, and where testing fits. ISO12207 7.2.5.3.2 is the required paragraph, which details (and uses the word) Testing - in effect (within ISO12207) Testing is synonymous with Validation

Improve this answer
3
  • Andrew, I do feel it the same way, yet the contradictions in other standards makes me confused. If you check that IEEE, it verifies/validates each artifact, not saying anything about testing. But as I mentioned earlier, the "requirement for intended use" is a bit vague term because as a tester, I can have just functional spec based on requirements which could be misunderstood. So the thing is what is the source for test cases?
    – John V
    Commented Nov 2, 2012 at 20:11
  • 1
    Hence my comment: This is the great thing about Standards - there are so many to choose from!
    – Andrew
    Commented Nov 3, 2012 at 13:30
  • Exactly. I want to justify the ISO. But how can I justify that unit testing is validation against intended use?
    – John V
    Commented Nov 3, 2012 at 13:47
4

"Testing", "validation", and "verification" are just words that people attach to different kinds of practices. Some of those practices focus more on "Did I build it right"; others focus more on "Did I build the right thing." You can name and group those practices in different ways depending on what you are trying to accomplish.

If you want to be clear about the terms you use, as you would in a scholarly work, you should decide upon the concepts you want to use and then attach words to those concepts. The particular words on not important as long as they are defensible. For example, I would not use the word "design" to collectively describe unit testing and integration testing because people are accustomed to "design" meaning something very different, but I could probably get away with calling both of those "verification" as long as I defined the term beforehand.

Improve this answer
2

Did I build it right (verification) and did we build the right thing (validation) is the best way to explain the difference to our new (and mature) engineers.

Remember that validation happens over the entire lifecycle. Acceptance Testing (verification) also contributes to the validation of the product. Once a product is accepted or bought off as meeting the customer and derived requirements in accordance with the agreed to test procedure then the product ownership becomes that of the customer.

In our business, the customer then performs Operational Test and Evaluation (OT&E) on the system before fleet fitment....this is purely validation at this point to see if it will do the job.

Improve this answer
1
  • @darrick: Acceptance testing is for sure validation, not verification. In acceptance test, the customer decides whether or not this is acceptable and fit for purpose.
    – John V
    Commented Nov 10, 2012 at 16:18
1

Simple difference in both Validation and Verification is: Validation is to check "did we build the right thing" or to check the expected results. Validation goes throughout the product lifecycle at different statges; from requirement to design to CAT(customer acceptance testing)

Verification is Validation of Validation. To find out whether something going wrong in processes being followed during lifecycle.

Improve this answer
1
  • Welcome to SQA&T. I like the idea of Ve being the Va of Va - except that Ve tends to happen before Va ;-)
    – Andrew
    Commented Nov 8, 2012 at 20:45
1

As user246 said, tell how you are going to use the terms and then use them that way. You have noticed that they are not well defined and even in standards there are contradictory definitions. Just tell which of those you are using.

You said:

When asking the famous questions, "Do I build it right" and "Did I build the right thing", it is not helpful from testing point of view.

I disagree with that. This is the most sensible way of separating verification and validation I have heard. From testing point of view this would mean the following:

  • Verification: Checking are the components of the program working correctly. Do the fields verify input, are dates handled correctly, are objects saved to the database, can we install it?
  • Validation: Asking can the user solve her problem with the software. Is it possible to enter all the data needed and reference it late, have we forgotten some essential part of the process, is the workflow logical, is it giving the value?

Of course it is not necessary to do this kind of distinction and often these things are tested at the same time. However, it might be a good mental model to have so that you don't forget one or the other.

Improve this answer
3
  • Apparently the ISO standard makes another division which might be as sensible. It's just on completely other level.
    – Edu
    Commented Nov 2, 2012 at 20:24
  • I did not make myself clear. The question "Do I build it right" is a good one but how do you know it is related to working instead of meeting requirements as stated in ISO? E.g. that code is testable, traceable to design, etc. Actually what you said would be validation because its testing whether it works as intended.
    – John V
    Commented Nov 2, 2012 at 20:46
  • It probably should apply to both definitions. The only way to know is to ask what the terms mean in certain context. I gave one interpretation, others have given others. I could add to my answer that not only tell which interpretation you are going to use, tell why you are using it.
    – Edu
    Commented Nov 2, 2012 at 22:17
0

Verification is verifying the expected behavior, but this doesn't make it valid. If you click on a widget, you can verify an action is triggered by that interaction. But it might bring up some UI that blocks other widgets or may not provide a UI that allows dismissal of the widget, thus blocking using the program. This would be invalid, as the user is blocked from further using the program.

I would tend to think of validation to include subjective evaluation.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.