Is Orthogonal Array Testing similar to Design of Experiments?

Question

I am studying software testing and I see this topic, in the past I was studying Six Sigma and I saw Design of Experiments, and these look similar to me.

Answer 1

No, not really.

Orthogoval Array Testing (OAT) is a way to combine inputs in order to have less number of test cases, which is mostly done to save time. An example could be when you have 3 variables and each can have 3 values. Traditionally you would generate 27 cases you need to test, with OAT it is down to 9.

Design of experiments, or experimental design, really is the basis for testing. The general idea behind it is that you design experiments in a controlled way in order to find more about your hypothesis, which usually states a causal relationship between independent variable(s) (also called factors) and dependent variables. This is different from observational study when you do not interfere, i.e. do not deliberately change any variables.

A simple experimental design could be the following one:

M => A => M
M    =>   M

Where M stands for measurement, A for administration of a treatment. So, you have two groups, one of which receives a treatment. You measure the dependent variable before administering the treatment, and after. A concrete example could be when you have two groups of patients and you wonder what effect a particular medication can have on their blood pressure. You divide them into two groups, one of which receives the medication, the other does not (e.g. receives a placebo).

Obviously it might get more complicated because you ideally want some randomization in your experiemnts as well, or you might further split the groups into e.g. age groups because you expect that the medication might have a different effect on different age groups. That's when other experimental designs come in.

Last but not least, ethics plays a role as well. Many times, true randomization is not possible because of ethics. E.g. you can't perform an experiement when you randomly divide people into groups when some of them are supposed to smoke; you can't force people to start smoking just because of your experiment.

All in all, key points are:

1. define your hypothesis
2. change one independent variable at a time
3. observe and measure the effect

What I see often is testers who change so many things in the system at one time and then go test it. Since they changed so many variables at a time, they can't really say afterwards which of those variables had a particular effect.

---

There are a number of resources about this on the Internet, a short introduction could be this article: http://www.stat.yale.edu/Courses/1997-98/101/expdes.htm but you can find tons of others.