166 reputation
2
bio website pitest.org
location
age
visits member for 1 years, 4 months
seen Mar 27 at 10:42

Nov
27
comment Mutation testing: is it useful for assessing test quality?
@Bj Rollison I agree that mutation testing is a poor fit for integration/system testing. Other forms of fault injection might add value there, but mutation testing requires tests that are both fast and highly repeatable (ie no dependency on external resources). Integration tests clearly don't fall into this category.
Nov
26
comment Mutation testing: is it useful for assessing test quality?
@dzieciou I think the value of equivalent mutations is a very interesting area. As I said, it has been my experience that a subset of them can be eliminated by improving the code - so they may be a marker for code smells. I don't know how true this is across all code bases/coding styles, and what constitutes good code is of course subjective. Hopefully there will be some research in this area.
Nov
26
comment Mutation testing: is it useful for assessing test quality?
@dzieciou I'm afraid I don't have any case studies (but it certainly would be a good idea to collect some and add them to the pitest site). You could however look at orfjackal.net. Esko has been using pitest for some time and I believe he has some screen casts showing development with it which might include some examples.
Nov
26
comment Mutation testing: is it useful for assessing test quality?
In terms of how much time it took to assess the surviving mutations . . . difficult to say. This was generally part of the normal development process and not measured separately. This is the point I was trying to drive home above, applying it as an after the fact quality assessment phase is going to be difficult. Where I've had success it has been part of the development process.
Nov
26
comment Mutation testing: is it useful for assessing test quality?
Yes, several real projects as described above. The main point of mutation testing isn't to find defects - it's to assess the quality of a test suite. I've found lots of test suite deficiencies with mutation testing and lots of unnecessary code. I have however found some defects via mutation testing - mainly incorrect conditional boundaries.
Nov
26
awarded  Teacher
Nov
26
awarded  Supporter
Nov
26
answered Mutation testing: is it useful for assessing test quality?