How machine learning can help in predicting defect prone modules in applications before testing?

Question

I have looked and read in the internet that using static code analysis data, we can predict what modules in the application are defect prone. When I try to imagine of dataset,it is getting difficult to fit it as 2 dimensional. Since each build can have many modules and for each module we will be having source code metrics and defects count. My doubts are

1. Can we some how build a meaningful 2D dataset for analysis (like regression) and prediction ?
2. Is it just a theory or machine learning is being actually leveraged for this ? If yes, what techniques/use cases are currently under use ?
3. What are the other practical factors that needs to be considered for defect prediction analysis and can we collect them in reality ?

It will be very helpful if you could point out some detailed articles on this,if any.

PS: I am posting this here because people with great expertise in testing can only advise on where to look at for defects.

Answer 1

Have a graph with number of bugs on the left vertical y axis

I would then have lines for factors that are relevant, e.g. lines of code, number of statements, line length, number of variables, global variables, etc. You could also have a formula that multiplies them together to give one line.

I would stay away from trying to do multi-dimensional models unless you have the experience and knowledge needed and have already done this with a 2d model. The details for 3d are important and don't fit well into a Stack Exchange question.

Answer 2

Thoughts on error-prone modules detection

One line of "defect prone modules" analysis may use the code complexity analysis to classify a module as "error-prone" or not. This would be based on the fact that a more complex code tend to have more potential bugs.

There are several metrics in the code complexity analysis that we can use as parameters for machine learning algorithms:

• Cyclomatic code complexity (What does the 'cyclomatic complexity' of my code mean?):

  Cyclomatic complexity is a software metric (measurement) which is a quantitative measure of the number of linearly independent paths through a program's source code.

• Maintainability Index

  Maintainability Index calculates an index value between 0 and 100 that represents the relative ease of maintaining the code

• Lines of Code

  A very high count might indicate that a type or method is trying to do too much work and should be split up. It might also indicate that the type or method might be hard to maintain.

Existing projects

According to Bug prediction at Google article, they've been successful in using the modified "Rahman" algorithm that was based on the number of times a module has been changed:

We implemented the Rahman algorithm by creating a program that hooked into our source control system, and pulls out all the changes which had a bug attached to them. It looks at each bug number, and verifies with the bug-tracking database that it was really a bug, and filters out everything else, such as feature requests. It then looks at all the files that appeared in these changes, and filters out those that have been deleted and are no longer at HEAD. For each file, the number of bug-fixing changes it's been in is calculated, and we output the files which were ranked in the top 10%.

This article became an inspiration for the bugspots and hotfiles projects.

There is also FixCache project (not maintained).

There is also the MAST group (Machine learning techniques for the Analysis of Source code Text) which published several machine learning models and algorithms for source-code analysis. I don't see something specific to detecting tendencies to errors in source code, but there are some interesting and related algorithms.

More articles on the subject:

• Use of Relative Code Churn Measures to Predict System Defect Density (there are also a lot of articles mentioned and linked in the paper)
• Static code analysis versus machine learning defect prediction