Now and then I run a test of all combinations of a test of variables. The outcome of each test is binary (e.g. pass/fail). Typically, some combinations will pass and others will fail. After the test is over, I want to look for patterns in the results.
An obvious way to look for results is to visualize them. For example, I might load the results into a spreadsheet and experiment with sorting various combinations of columns. After sorting, if the outcome column contains a long sequence of the same value, I may have found a pattern.
An alternative would be to let the computer look for patterns. I envision a command line program that takes a matrix of combinations and corresponding outcomes as input and produces a set of rules as output, e.g. "always passes when variable-1 == 3", or "always fails when variable-2 == 4 and variable-3 == 'b'".
I think I know how I would go about writing something like that, but surely this has been done before. Is there any open-source software available for summarizing combinatorial test results?
(1,1,0) => failure
, based on the statistical data you have from your tests. – dzieciou Jan 26 '14 at 22:11