You are browsing the archive for pit.

Android Mutation Testing – Back from AnDevCon SFO 2016

December 9, 2016 in Android

I just get back from AnDevCon that was be held from 29 November to 01 December @San Fransisco, USA. The conference organization was good and I had the chance to meet community Android folks.

In AnDevCon 2016, I had the chance to talk about “Mutation Testing for Android” in 01 December.

My session video is uploaded below.

My session slides is uploaded below.

Speaking in AnDevCon SFO 2016

November 24, 2016 in Android, Conference

adc-sf2016-header

@Thursday, 01 December 2:15 pm – 3:30 pm, I will be speaking in AnDevCon SFO conference (that will be held from 29 November to 01 December @San Francisco, USA) about “Android Mutation Testing”. In my session, I will discuss unit testing frameworks in Android, traditional code coverage techniques and mutation testing using PIT. The session will be a practical one, it will have two demos:
http://www.andevcon.com/sf2016/speakers#HazemSaleh

At the end of the session, there is an award for the one who will solve a quiz.

I really wish to see all of you there in AnDevCon SFO 2016!

Dagger 2, MVP and Unit Testing (PIT Mutation Testing) – Part 4

September 25, 2016 in Android, Java, Unit Testing

The previous three articles[1] showed an introduction to Dagger, MVP pattern, Espresso and more. This article focuses on introducing a powerful test coverage technique for ensuring that the app unit tests are strong enough.

Introduction to Mutation Testing

One of the golden standard test coverage techniques is mutation testing. The main idea of mutation testing is to perform byte code modifications (mutations) to your original app source code and then run your app unit tests to check if they are strong enough to fail as a result of these mutations.

The byte code changes in Mutation Testing are named as faults (or mutations). After applying the source code changes, if your tests fail then the mutation is killed, and if your tests pass then the mutation is lived.

So in summary, If your tests fail by source code mutations then this means that your tests are really strong, if your tests pass then this means you need to modify your tests to make them strong enough (The quality of your tests can be gauged from the percentage of mutations killed).

Why we need Mutation Testing

Mutation tests have an advantage over Traditional test coverage (i.e line, statement, branch etc) because it does not only measure which code is executed by your tests but also it checks that your tests are actually able to detect faults in the executed code. This gives a strong measurement to your developed unit tests.

PIT

PIT is one of the mutation testing systems for Java. It has several advantages:

  1. Fast to execute.
  2. Integrated with Java build tools (Ant, Maven and Gradle).
  3. Active.

More information about PIT can be found in:
http://pitest.org/

Android Integration

Unfortunately, the official PIT Gradle plugin does not work fine with Android Gradle projects:
https://github.com/szpak/gradle-pitest-plugin

However, thanks to one of the community member, this support is now possible using this fork:
https://github.com/koral–/gradle-pitest-plugin

Application Configuration

In order to configure Android PIT plugin with our Android Dagger app:
1. Add PIT plugin to your top-level build.gradle file:

classpath 'pl.droidsonroids.gradle:gradle-pitest-plugin:0.0.3'

2. Configure PIT plugin in your app build.gradle file:

pitest {
   targetClasses = ['com.test.xyz.daggersample1.ui.presenter.*']  /* specify target classes to be mutated */
   excludedClasses = ['**Factory*'] /* Exclude Dagger generated classes */
   pitestVersion = "1.1.0" 
   threads = 4 /* specify number of threads */
   outputFormats = ['XML', 'HTML'] /* specify output format */
}

Checking Results

After configuration, finally, we can execute PIT tests by running:

./gradlew pitest

You can pick the PIT tests report under build/reports/pitest.

The following figure shows the mutation testing report in our Dagger sample:
PIT Report

The following figure shows survived and killed mutants:
Mutants

FYI, in the mutation report, Light green shows line coverage, dark green shows mutation coverage, Light pink show lack of line coverage, and dark pink shows lack of mutation coverage.

Source code

App Source code is available in:
https://github.com/hazems/Dagger-Sample/tree/pit1/app/src/test/java/com/test/xyz/daggersample1/ui/presenter

Top-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/build.gradle

Module-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/app/build.gradle

In our next article, let’s see how to kill these survived mutants which makes our tests not strong enough!

[1] Previous articles:
http://www.technicaladvices.com/2016/03/05/an-introduction-to-dagger-2-android-di-part-1/
http://www.technicaladvices.com/2016/03/09/an-introduction-to-dagger-2-android-di-part-2/
http://www.technicaladvices.com/2016/04/07/dagger-2-mvp-and-unit-testing-android-di-part-3/

Skip to toolbar