Being inspired by the question "Good practices for identifying "one-time" tests?" I decided to make a proof of concept for the idea of applying supervised machine learning algorithms to a problem of "one-time-test" classification. The question such the classification would answer is When one creates a test, what class would that test be assigned? Whether that class would be a "one-time test" or not?.
In my idea that aproach would not set an exact instruction to remove a test after the new version of tested product is been released. It would rather flag a test to help a test manager revise it after some reasonable time to decide if it really needs to be removed.
In my comment I suggested to build up a Neural Network, however NN is not the only paradigm that can be used. Without respect to the particular classification algorithm, all supervised algorithms require a training data set that is composed of a set of "features" and the actual class that the feature vector (basically it means a set of test properties) represents.
Thus there are two problems I have to solve. They are:
- What is a "one-time test" (it is rather not a problem but a question)
- How to define the appropriate feature-vector.
Regarding the first question I would define a one-time test as: "a test that is executed for only a single release on the extent of, say, 5 sequential major releases". The second issue is actually what I would like to get your advice for. This might be considered as a dup of the question that I've been inspired with, however this one emphasizes the countability of the test properties and rather the objective analysis of the test properties. Yet, I have come up with the following (considering the properties should be countable) features (which I think impact the class the test would be assigned to):
- Number of test steps
- Estimated execution time
- Number of high priority tests the test designer has introduced overall
- Number of low priority tests the test designer has introduced overall
- Number of reworks the tests designed by the particular test designer underwent
- Ratio of one-time-to-non-one-time tests the test designer has introduced overall (this one and three previous features is the attempt to objectively define subjective test designer competence)
- Number of letters in test description (since that could define how concrete the tester is)
- The number of the day in a year the test is created (since there are some days when a test designer might put less effort to their job as they would usually do)
- Number of the tests for this particular component that exist. (Represents how well the component is covered with the tests so far)
Hence my question is:
What countable test properties (which could be picked from a given regression historical test set) could potentially impact the test reusability rate?