8

Lately I have been researching Machine Learning, particularly Genetic Algorithm. The principles from what I can tell are rather straightforward and simple.

One of the basics is that it attempts to determine the best result by using a massive amount of data. The issue I am having is that I need to quantify the results however with SQA the results are usually binary (Pass or Fail) with very little area for wiggle room.

Does anyone have any experience with quantifying results beyond pass or fail?

I was considering using machine learning to optimize test runs, in essence to say 'Using x data exercises 20% more of the application than y', or potentially to attempt to achieve 100% code coverage through unit tests.

To ask a clear question: Is there any way to quantify the results from a test beyond pass or fail, if so, how have you either implemented this or seen this implemented?

Improve this question
3
  • 1
    Is your goal to use machine learning, or is your goal to optimize test runs? I ask because it isn't obvious that you need to use machine learning to optimize test runs.
    – user246
    Commented Dec 22, 2016 at 16:17
  • Well, to be honest, the prior. I am more interested in the potential of implementing the technology into tools in the field than any particular tool.
    – Paul Muir
    Commented Dec 22, 2016 at 19:11
  • And any potential for impact that such tools can have on the field but that is another question for another time.
    – Paul Muir
    Commented Dec 22, 2016 at 19:24

2 Answers 2

Reset to default
4

There're several test tools that already use machine learning. Especially within the field of search-based software testing (SBST), genetic algorithms (GAs) are often used to optimize test generation towards different goals, such as code coverage or test length. Some examples are:

Search for multi-objective search-based testing (MoSBaT), there you'll find more information on how you can "quantify" test suites/cases.

Other interesting approaches using machine learning for software testing are:

Improve this answer
1

At the end of the day you need to make a binary decision, can the code under test be shipped to its customers ?

But the way to this decision can vary, performance tests are a good example- in some places the definition is something like "better than last N version(s)" or "no major degradation"

Other than that you can use the results as guideline to decision makers using comparative graphs or numbers, maybe even using ML across versions to identify trends or issues.

Improve this answer

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge you have read our privacy policy.

Not the answer you're looking for? Browse other questions tagged or ask your own question.