2

I am researching what the best techniques are in generating test cases or regression suites using machine learning for black box testing.

Has anyone researched previously or able to help?

  • Black box testing? Like end-user experience testing? – Yu Zhang Jul 16 '18 at 23:43
2

You can apply Neural Network conception for this problem. You should only consider that the accuracy of any machine learning application depends on the amount of data you use to train your algorithm.

To do that you should formalize your problem. NNs are intended to solve classification problems which usually answer the question "if an object belongs to the particular class".

Say, you can define a class "A test suitable for regression for this particular change".

Now you should prepare a training set which would consist of the input and the output that is presumed correct. Here one of the main issues is how you formalize the input.

While training you will basically feed your network with a number of train data entries. Each entry might represent the following:

some formal representation of the change (you will have to break down "a change" to a number of formal properties, flags, numbers, etc) concatenated with some formal representation of the test that fits regression class for the change encoded in that particular train data entry.

Thus, your data would look like:

CHNG_PROP1, CHNG_PROP2, .., CHNG_PROPN, TEST_PROP1, TEST_PROP2, .., TEST_PROP_N, GOOD_FOR_REGR
some_val,some_val,..,some_val,,some_val,,some_val,..,some_val,Y
some_val,some_val,..,some_val,,some_val,,some_val,..,some_val,N
some_val,some_val,..,some_val,,some_val,,some_val,..,some_val,N
some_val,some_val,..,some_val,,some_val,,some_val,..,some_val,Y
...
some_val,some_val,..,some_val,,some_val,,some_val,..,some_val,Y

Then for some new change you will:

  • Break your new change and all your tests to set of properties, flags, numbers, etc. like you did for your training set
  • Run your NN for that new change and for each of the test you have

NN gives you the answers in terms of probability so you will get the list of probabilities (one for each test) saying how well the particular test fits your problem.

Then you just take the tests which produce the probability more than a threashold and hence you're regression set is built.

| improve this answer | |
  • 2
    This is a reasonable description of a general technique for training a neural network. There are three kinds of information that the OP will need to work out: (1) a tuple space that describes any kind of code you might want to unit test, or alternatively, any kind of change you might make to code that you want to unit test; (2) a tuple space describing unit tests that you might want to generate; and (3) a data set with lots of examples of successful and unsuccessful (code under test, unit test) pairings to use a training data. – user246 Jul 16 '18 at 15:24

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.