The previous three articles showed an introduction to Dagger, MVP pattern, Espresso and more. This article focuses on introducing a powerful test coverage technique for ensuring that the app unit tests are strong enough.
Introduction to Mutation Testing
One of the golden standard test coverage techniques is mutation testing. The main idea of mutation testing is to perform byte code modifications (mutations) to your original app source code and then run your app unit tests to check if they are strong enough to fail as a result of these mutations.
The byte code changes in Mutation Testing are named as faults (or mutations). After applying the source code changes, if your tests fail then the mutation is killed, and if your tests pass then the mutation is lived.
So in summary, If your tests fail by source code mutations then this means that your tests are really strong, if your tests pass then this means you need to modify your tests to make them strong enough (The quality of your tests can be gauged from the percentage of mutations killed).
Why we need Mutation Testing
Mutation tests have an advantage over Traditional test coverage (i.e line, statement, branch etc) because it does not only measure which code is executed by your tests but also it checks that your tests are actually able to detect faults in the executed code. This gives a strong measurement to your developed unit tests.
PIT is one of the mutation testing systems for Java. It has several advantages:
- Fast to execute.
- Integrated with Java build tools (Ant, Maven and Gradle).
More information about PIT can be found in:
Unfortunately, the official PIT Gradle plugin does not work fine with Android Gradle projects:
However, thanks to one of the community member, this support is now possible using this fork:
In order to configure Android PIT plugin with our Android Dagger app:
1. Add PIT plugin to your top-level
2. Configure PIT plugin in your app
targetClasses = ['com.test.xyz.daggersample1.ui.presenter.*'] /* specify target classes to be mutated */
excludedClasses = ['**Factory*'] /* Exclude Dagger generated classes */
pitestVersion = "1.1.0"
threads = 4 /* specify number of threads */
outputFormats = ['XML', 'HTML'] /* specify output format */
After configuration, finally, we can execute PIT tests by running:
You can pick the PIT tests report under
The following figure shows the mutation testing report in our Dagger sample:
The following figure shows survived and killed mutants:
FYI, in the mutation report, Light green shows line coverage, dark green shows mutation coverage, Light pink show lack of line coverage, and dark pink shows lack of mutation coverage.
App Source code is available in:
Top-level Gradle file PIT configuration:
Module-level Gradle file PIT configuration:
In our next article, let’s see how to kill these survived mutants which makes our tests not strong enough!
 Previous articles: