Mockito 2.x over PowerMock Migration Tips and Tricks (Top Ten)

After so many years of hopeless waiting, Mockito 2.x is released to solve many problems that most of the android developers were having in their tests.

There are great features of Mockito 2.x which includes:

  1. Finally mocking final classes and methods.
  2. Support for Java 8.
  3. Migration from CGLIB to ByteBuddy.

But if you are having today large tests written in Mockito 1.x, will it be an easy task to migrate?

Unfortunately, the migration most probably will be a painful task because Mockito 2.x does not respect the old behavior of Mockito 1.x. Adding to this complexity, If you are having PowerMock in your old tests, then you will have to face another dimension of complexity since most of PowerMock’s versions are having integration issues with Mockito 2.x.

Regarding PowerMock’s early issues with Mockito 2.x, PowerMock team announced that PowerMock 1.6.5 has an experimental support for Mockito 2.x but unfortunately it was not that great. It has some problems with Mockito 2.x.

In the beginning when just changing Mockito version to 2.x in your build.gradle file, you may find more than 50% of your tests were failing due to a lot of issues, Null pointer exceptions, compilation errors, No class definition found, un-expected thrown exception, …etc, and this is how you may look in the beginning of the migration.

Do not panic and do not be sad, this blog post mentions some of the important challenges that you may face during the migration and tips to overcome these challenges to save your time.

1. Use the proper PowerMock’s Mockito API extension

Using powermock-api-mockito extension does not work with Mockito 2.x, you will have the following exception when running your unit tests if you stick to the old extension:

java.lang.NoClassDefFoundError: org/mockito/cglib/proxy/MethodInterceptor
at org.powermock.api.mockito.internal.mockmaker.PowerMockMaker.<init>(PowerMockMaker.java:43)
...
Caused by: java.lang.ClassNotFoundException: org.mockito.cglib.proxy.MethodInterceptor
at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
... 40 more

In order to fix this issue, you should use the right Mockito’s API extension which is:
powermock-api-mockito2.

2. Avoid using Incompatible versions of Mockito and PowerMock

Always make sure to use compatible versions of Mockito and PowerMock, for example the following two versions are compatible:

  • PowerMock version 1.7.0 RC2.
  • Mockito version 2.1.0.

3. Say Goodbye to Mockito’s Whitebox

Mockito 2.x does not have Whitebox anymore.

So what is the solution then?

Initially, you can use PowerMock’s Whitebox instead of the removed Mockito’s 2.x Whitebox. However, you need to know that this does not come without problems, one of the problem which I reported to PowerMock’s issues is (org.powermock.reflect.exceptions.FieldNotFoundException: No instance field named XXX could be found in the class hierarchy):
https://github.com/powermock/powermock/issues/773

So if the initial solution does not work with you, why not writing your own one, it is not really hard.

4. Using the Right Matchers

Never forget to always use org.mockito.ArgumentMatchers instead of the old org.mockito.Matchers.

For example, replace the old matcher imports:

import static org.mockito.Matchers.any;
import static org.mockito.Matchers.anyBoolean;
import static org.mockito.Matchers.anyInt;
import static org.mockito.Matchers.anyString;
import static org.mockito.Matchers.eq;

With the following ones:

import static org.mockito.ArgumentMatchers.any;
import static org.mockito.ArgumentMatchers.anyBoolean;
import static org.mockito.ArgumentMatchers.anyInt;
import static org.mockito.ArgumentMatchers.anyString;
import static org.mockito.ArgumentMatchers.eq;

And so on with other matchers.

5. anyInt() does not match Long literals anymore

After the upgrade, you may find anyInt() does not work because it cannot match long literals such as 0L for example :).

So in order to fix this issue, just replace anyInt() with anyLong() and you will be fine.

OT note, if you can match with the actual value instead of anyXXX(), this can be much better and will give your test more transparency.

6. anyString() does not now match null anymore

This can fail many tests, anyString() now does not include null anymore in Mockito 2.x. This applies also to any(xxx), for example, any(InputStream.class) does not now match null in Mockito 2.x as well.

Again OT note, if you can match with the actual value instead of anyXXX(), this can be much better and will give your test more transparency.

7. Mockito 2.7.1 is not a real friend to PowerMock

Unfortunately, if you use PowerMock 1.6.5 or even PowerMock 1.7.0RC2 with Mockito 2.7.1 (the latest version at the time of writing this post), you will find the following exception with donothing:

java.lang.IllegalAccessError: 
tried to access method 
org.mockito.internal.stubbing.answers.DoesNothing.<init>()V from 
class org.powermock.api.mockito.PowerMockito

Fortunately, this issue is fixed with PowerMock 1.7.0RC4, and below is the fix issue URL for your information:
https://github.com/powermock/powermock/issues/753

So in summary if you use Mockito 2.7.1, do not forget to use PowerMock 1.7.0RC4.

8. Original test exceptions are wrapped as RuntimeExceptionProxy in Mockito 2.x with PowerMock

Unfortunately as a workaround, you have to modify all the broken @Test(expected=SomeException.class) to @Test(expected=Exception.class) since original exceptions are wrapped as Mockito RuntimeExceptionProxy in Mockito 2.x with PowerMock.

This issue really requires further investigation to know why Mockito 2.x does this wrapping with PowerMock. if you have a better solution for this, feel free to comment to the post.

9. Move away from PowerMock and depend on Mockito 2.x only

Try to create a plan to remove PowerMock by refactoring your app classes to be testable. Mockito 2.x is really enough now.

10. Review old tests

Take this migration as a chance to review the old tests and to improve them in order to have a better maintainable tests. This can help you strengthen your product code and allow easier refactoring for the current code base without surprises.

Android Mutation Testing – Back from AnDevCon SFO 2016

I just get back from AnDevCon that was be held from 29 November to 01 December @San Fransisco, USA. The conference organization was good and I had the chance to meet community Android folks.

In AnDevCon 2016, I had the chance to talk about “Mutation Testing for Android” in 01 December.

My session video is uploaded below.

My session slides is uploaded below.

Speaking in AnDevCon SFO 2016

adc-sf2016-header

@Thursday, 01 December 2:15 pm – 3:30 pm, I will be speaking in AnDevCon SFO conference (that will be held from 29 November to 01 December @San Francisco, USA) about “Android Mutation Testing”. In my session, I will discuss unit testing frameworks in Android, traditional code coverage techniques and mutation testing using PIT. The session will be a practical one, it will have two demos:
http://www.andevcon.com/sf2016/speakers#HazemSaleh

At the end of the session, there is an award for the one who will solve a quiz.

I really wish to see all of you there in AnDevCon SFO 2016!

Dagger 2, MVP and Unit Testing (PIT Mutation Testing) – Part 4

The previous three articles[1] showed an introduction to Dagger, MVP pattern, Espresso and more. This article focuses on introducing a powerful test coverage technique for ensuring that the app unit tests are strong enough.

Introduction to Mutation Testing

One of the golden standard test coverage techniques is mutation testing. The main idea of mutation testing is to perform byte code modifications (mutations) to your original app source code and then run your app unit tests to check if they are strong enough to fail as a result of these mutations.

The byte code changes in Mutation Testing are named as faults (or mutations). After applying the source code changes, if your tests fail then the mutation is killed, and if your tests pass then the mutation is lived.

So in summary, If your tests fail by source code mutations then this means that your tests are really strong, if your tests pass then this means you need to modify your tests to make them strong enough (The quality of your tests can be gauged from the percentage of mutations killed).

Why we need Mutation Testing

Mutation tests have an advantage over Traditional test coverage (i.e line, statement, branch etc) because it does not only measure which code is executed by your tests but also it checks that your tests are actually able to detect faults in the executed code. This gives a strong measurement to your developed unit tests.

PIT

PIT is one of the mutation testing systems for Java. It has several advantages:

  1. Fast to execute.
  2. Integrated with Java build tools (Ant, Maven and Gradle).
  3. Active.

More information about PIT can be found in:
http://pitest.org/

Android Integration

Unfortunately, the official PIT Gradle plugin does not work fine with Android Gradle projects:
https://github.com/szpak/gradle-pitest-plugin

However, thanks to one of the community member, this support is now possible using this fork:
https://github.com/koral–/gradle-pitest-plugin

Application Configuration

In order to configure Android PIT plugin with our Android Dagger app:
1. Add PIT plugin to your top-level build.gradle file:

classpath 'pl.droidsonroids.gradle:gradle-pitest-plugin:0.0.3'

2. Configure PIT plugin in your app build.gradle file:

pitest {
   targetClasses = ['com.test.xyz.daggersample1.ui.presenter.*']  /* specify target classes to be mutated */
   excludedClasses = ['**Factory*'] /* Exclude Dagger generated classes */
   pitestVersion = "1.1.0" 
   threads = 4 /* specify number of threads */
   outputFormats = ['XML', 'HTML'] /* specify output format */
}

Checking Results

After configuration, finally, we can execute PIT tests by running:

./gradlew pitest

You can pick the PIT tests report under build/reports/pitest.

The following figure shows the mutation testing report in our Dagger sample:
PIT Report

The following figure shows survived and killed mutants:
Mutants

FYI, in the mutation report, Light green shows line coverage, dark green shows mutation coverage, Light pink show lack of line coverage, and dark pink shows lack of mutation coverage.

Source code

App Source code is available in:
https://github.com/hazems/Dagger-Sample/tree/pit1/app/src/test/java/com/test/xyz/daggersample1/ui/presenter

Top-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/build.gradle

Module-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/app/build.gradle

In our next article, let’s see how to kill these survived mutants which makes our tests not strong enough!

[1] Previous articles:
http://www.technicaladvices.com/2016/03/05/an-introduction-to-dagger-2-android-di-part-1/
http://www.technicaladvices.com/2016/03/09/an-introduction-to-dagger-2-android-di-part-2/
http://www.technicaladvices.com/2016/04/07/dagger-2-mvp-and-unit-testing-android-di-part-3/

Back from JavaOne China 2013

JavaOne China

JavaOne China

I just get back from JavaOne China that was held in Shanghai 22-25 July 2013. JavaOne China is one of the biggest Java Conferences in Asia. The conference organization was good and there were a lot of attendees in the conference sessions.

JavaOne China

JavaOne China

I had the chance to present “Efficient JavaScript Unit Testing” in 23 July, the room of the session was full of attendees and I was really amazed by the energy, enthusiasm, and responsiveness of the attendees during the session.

Photo with Chinese Developers

Photo with Chinese Developers

The session went great and I was glad to give a free copy of my JavaScript Unit Testing book to one of the attendees who answered a JavaScript quiz at the end of the session.

I uploaded my session below:

Aside from the conference, Shanghai is a modern city that has many good places to visit, I had the chance to visit three places there:
1. Yuyuan Garden and Old Town Area:
Yuyuan Garden Photos

2. Jing Temple:
Jingle Temple Photos

3. Oriental Pearl TV Tower:
Oriental Pearl TV Tower Photos

“Efficient JavaScript unit testing” video session in CONFESS 2012

This video provides a presentation about how to make efficient JavaScript unit testing. I presented it in the CON-FESS 2012 international conference which was held in Leogang, Austria. I wish it can be useful.

See the presentation with the slides on Parleys.com

See the presentation with the slides on Parleys.com

and I included below the presentation:

“Efficient JavaScript unit testing” presentation in CONFESS 2012

I just got back from the International technical conference CONFESS 2012 which was held in Leogang, Austria. The conference was very cool, the hotel and the city were fantastic, the quality of the sessions and the speakers were great.

Now, I got finally some free time to share with you my presentation about my efficient JavaScript unit testing. I hope it will be useful for you. Once I get the video session, I will definitely share it with you as well.

Have a nice time.

JsTestDriver integration with Hudson

Abstract

JsTestDriver is one of the most efficient JavaScript unit testing tools. Integrating JsTestDriver with continuous integration tools is a mandatory step for ensuring the web application quality. In this article, I will illustrate how this sort of integration can be done between JsTestDriver and Hudson.

Installing Hudson

To install Hudson on your machine, you need to download it from http://hudson-ci.org/.

Run the following command to start Hudson:

java -jar hudson-2.2.0.war

You can start launching Hudson by typing the following address in your browser:

http://localhost:8080/

You should now be redirected to the Hudson home page.

Creating Hudson Job

First of all, you need to create a new Job in Hudson. You can do this by clicking on “New Job” and entering the Job name and selecting “Build a free-style software project” as shown in the figure below:

Configuring JsTestDriver with Hudson

Click the “Configure” link of the project. In the configuration page, enter the project description. In order to run the job periodically, you need to configure the “Build Triggers”. Check the “Build periodically” option and enter the cron-like expression. To run the job every 5 mins past the hour (for example), enter the following expression 5 * * * * as shown below in the figure:

In order to execute the JsTestDriver test command, you need to configure the “Build” section. There are many ways to do so, you can either add “Execute Shell”, “Invoke Ant or Maven2 or Maven3”, or “Execute Windows Batch command”. The latter case can be used when you are working on Windows environment. For the case of this example, I use the “Execute Windows Batch command” option and specify the path of the batch file as shown in the screenshot below:

The batch file has the following content:

java -jar JsTestDriver-1.3.4.b.jar 
      --config jsTestDriver.conf 
      --tests all 
      --testOutput c:/outputFolder 
      --reset

For the matter of simplicity, I assume that the JsTestDriver server is already started and some browsers are already captured. This step is not included in this post, however, This step also can be automated if required.

Testing the integration of JsTestDriver with Hudson

In order to test the integration of JsTestDriver with Hudson, this can be done by manually or automatically (as indicated in the “Build periodically” option). To test the integration manually, click on the “Build Now” link as shown in the screenshot below:

Click the “console output” link to check the output of the results:

setting runnermode QUIET
Firefox: Reset
...
Total 3 tests (Passed: 3; Fails: 0; Errors: 0) (3.00 ms)
  Firefox 12.0 Windows: Run 3 tests (Passed: 3; Fails: 0; Errors 0) (3.00 ms)

Notices

You may face this error while trying to run a new Job:

hudson java.io.IOException: Cannot run program "sh"

In order to fix this error, check this post:
http://www.technicaladvices.com/2012/05/04/hudson-java-io-ioexception-cannot-run-program-sh-resolution/.

More Information about JsTestDriver

Efficient JavaScript unit testing

Generating Test Reports using JsTestDriver

Running Jasmine on the top of the JsTestDriver test runner

Testing JavaScript code using Jasmine (Kickstart)

Jasmine is one of the JavaScript unit testing frameworks. It has a nice advantage that its syntax is readable and a very human friendly. In this post, I will give you a kickstart to help you start working with Jasmine.

First of all, Let’s create a simple JavaScript object that we need to perform some unit testing on it:

/* Basics.js file */
function Basics() {
}

Basics.prototype.add = function(x, y) {
	return x + y;
};

As shown in the Basics.js file, Basics is a simple JavaScript object that contains one method that adds the x and y parameters and finally returns the result to the caller.

To start working with the Jasmine framework, download the framework from http://pivotal.github.com/jasmine/download.html. Make sure that you have the following folder structure after extracting the downloaded zip file:
jasmine structure

You are having three folders and one html file.
1. lib folder: contains the source files of the Jasmine framework.
2. src folder: contains the source JavaScript files that will be tested.
3. spec folder: contains the testing JavaScript files.
4. SpecRunner.html file: is used for running the JavaScript test suites.

Place the Basics.js in the src folder after making sure that you remove all the old contents of the src and the spec folders.

Now, Let’s write the test suite in the BasicsSpec.js file.

describe("Basics", function() {
  var basics = new Basics();  

  it("should be able to make correct addition", function() {
    expect(basics.add(3, 2)).toEqual(5);
  });

});

As we see here, the testing syntax is very human friendly, it means “describe Basics object which should be able to make correct addition”. The describe keyword represents a test suite that can contain one or more specs. Every spec is represented by the it keyword.

Inside the spec, there is an expectation that expects the basics.add(3, 2) is equal to 5. In Jasmine, the expectation is represented by the expect keyword and the equality checking is represented by a matcher called toEquals.

In Jasmine, there are many other built-in matchers; you can find all of them here: https://github.com/pivotal/jasmine/wiki/Matchers.

Finally, do not forget to remove the old source and spec JS includes from the SpecRunner.html and to add the new includes instead:

  <!-- include source files here... -->
  <script type="text/javascript" src="src/Basics.js"></script>  
  
  <!-- include spec files here... -->
  <script type="text/javascript" src="spec/BasicsSpec.js"></script>

After running the SpecRunner.html file, you will find the following screenshot telling you that the test spec runs successfully.

screenshot

I wish this kickstart can be useful for you. For your reference, download the kickstart example from here.

Generating Test Reports using JsTestDriver

JsTestDriver is one of the most powerful JavaScript unit testing tools. I wrote an article in developerWorks about efficient JavaScript unit testing using JsTestDriver, please read first it before reading this post.

In this post, I will show you how to generate test coverage reports using the JsTestDriver. First of all, Let’s understand what is meant by “Test Coverage”. “Test Coverage” is one of the measures of software testing. It describes how much the source code of the program is tested.

The coverage criteria can be on the function level (Function coverage), on the statement level (Statement coverage), and on the code branch level (Branch coverage).

JsTestDriver has a plugin that can generate code coverage for the JavaScript code. To configure this plugin, you should:

  • Download the “coverage.jar” from http://code.google.com/p/js-test-driver/downloads/list
  • Add the coverage plugin declaration to the configuration file (jsTestDriver.conf file) as follows:

    ...
    plugin: 
    - name: "coverage" 
      jar: "plugins/coverage.jar" 
      module: "com.google.jstestdriver.coverage.CoverageModule"
    
  • Specify the --testOutput [output_folder] flag in the test running command. This will get the test coverage files in the output_folder you specified.

Currently, The JsTestDriver generates the test coverage files in LCOV format. Unfortunately, JsTestDriver does not generate HTML reports directly as shown in the figure below.
Genarated LCOV files

To generate the HTML reports from the LCOV files, you can use the LCOV visualizer, check it here: http://ltp.sourceforge.net/coverage/lcov.php.

The LCOV visualizer tool is working on Linux Redhat environment. To generate the report from the tool, execute this command on the jsTestDriver.conf-coverage.dat file:

genhtml jsTestDriver.conf-coverage.dat

For more information about how to install the LCOV visualizer on Linux Redhat environment, and converting the LCOV files to HTML reports:
http://www.technicaladvices.com/2012/05/02/converting-the-jstestdriver-coverage-files-to-html-reports/.

Finally for your reference, You can download the project from here, and the LCOV test coverage files from here.

I wish that this tip can be useful.