About mutation score after extract method refactoring

[surli]

We got a question regarding the mutation score and the mutation points that are considered in pit-descartes.

We perform a refactoring of one of our classes to split a big method in several ones, you can see it here: https://github.com/xwiki/xwiki-platform/pull/959/files#diff-a9a4498e3f9e2cc733735773f8843ed9

Before this refactoring, the method execute was containing almost the whole business of the class and it was very partially covered: only the first line was killed. see:

We obtained then 1 mutation killed on 4, so 25% for the mutation score on this file. For the record the coverage on this file was 8/55 so 15%.

After our refactoring, we now have many more methods, and we did not change the test. So the same mutation is still killed. See:

However, it now computes 1 mutation killed over 13, so it's now 8% for the mutation score which decreases a lot our threshold. FTR the coverage remains almost identical: 9/62 so still 15%.

So the question is, how exactly those mutation points are computed, and how should we handle such refactoring cases? Because basically for us we improved the code quality without decreasing our test quality: however our mutation score says the contrary since the number of mutation points increased.

[oscarlvp]

Descartes performs mutations at the method level. It means that it will create mutants by transforming the entire method body. Each mutant corresponds to a method variant created by Descartes. After creating more methods in the same class it should be expected to create more mutants. The number of mutants, or variants, for each method, depends on the return type, but never exceeds three mutants per method.

These reports come from the default HTML report format provided by PIT. It is not that the mutant is killed only in the first line. If you check the description of the mutant, it should say something like "All instructions replaced by return null" for the method in question, so the mutation expands to the entire body, not just the first line.

Now, it seems that the new mutants, created for the new methods are not covered. PIT will count these uncovered mutants as non-detected mutants when computing the score, that's why you get a lower score.

[oscarlvp]

See STAMP-project/pitmp-maven-plugin#42