What is the term for testing a system's behavior by comparing it to a system that is known to work correctly?

Question

For example, let's pretend that I want to write a program that encrypts data using a given algorithm.

Instead of manually calculating expected outputs and comparing them with actual outputs, I might instead compare the outputs produced by my program with outputs produced by OpenSSL.

There was a term for this but I can't remember. What is the name of this practice?

Answer 1

As mentioned in my comment, you are probably referring to a special type of test oracle.

João Farias' answer already got to the point: test oracles are used all the time. However, the type of the oracle varies. When you, e.g., write a unit test for a particular method:

// 1st param: expected, 2nd param: actual
assertEquals(3, mySumMethod(1, 2));

The expected output is typically determined by a human—the developer. But sometimes, you refer to a reference implementation:

assertEquals(referenceSumMethod(1, 2), mySumMethod(1, 2));

In contrast to the first variant, the human oracle, I would say this falls into the category of the derived oracle. As far as I'm aware, this is sometimes used in property-based testing with libs like QuickCheck, for example.

For further reading: In Douglas Hoffman's paper "A Taxonomy for Test Oracles", you find various classes of test oracles. But also Elaine J. Weyuker's original work on the oracle assumption ("On Testing Non-testable Programs") is a nice read.

Answer 2

I've often heard of this as regression testing.

• You know how things worked before and tests pass
• You make a change
• You test again and see if anything has regressed with bugs

Answer 3

It would say it depends on your goal with the test:

Portability testing and compatibility testing mean that your system will work on different environments.

However, your goal maybe to do security testing because you trust the encryption from OpenSSL.

You can also say you are doing automated test data generation or something like this, because you are externalizing completely the test data generation to computers.

About @beatngu13 answer: A test oracle is used all the time. Oracles are models to check the outputs. If you were to calculate the expected outputs, the oracle would be your mental model of the system. Using OpenSSL, your oracle is the OpenSSL model. That's why I think it doesn't answer properly the question.