You are browsing the archive for Unit Testing.

Android Mutation Testing – Back from AnDevCon SFO 2016

December 9, 2016 in Android

I just get back from AnDevCon that was be held from 29 November to 01 December @San Fransisco, USA. The conference organization was good and I had the chance to meet community Android folks.

In AnDevCon 2016, I had the chance to talk about “Mutation Testing for Android” in 01 December.

My session video is uploaded below.

My session slides is uploaded below.

Speaking in AnDevCon SFO 2016

November 24, 2016 in Android, Conference

adc-sf2016-header

@Thursday, 01 December 2:15 pm – 3:30 pm, I will be speaking in AnDevCon SFO conference (that will be held from 29 November to 01 December @San Francisco, USA) about “Android Mutation Testing”. In my session, I will discuss unit testing frameworks in Android, traditional code coverage techniques and mutation testing using PIT. The session will be a practical one, it will have two demos:
http://www.andevcon.com/sf2016/speakers#HazemSaleh

At the end of the session, there is an award for the one who will solve a quiz.

I really wish to see all of you there in AnDevCon SFO 2016!

Dagger 2, MVP and Unit Testing (PIT Mutation Testing) – Part 4

September 25, 2016 in Android, Java, Unit Testing

The previous three articles[1] showed an introduction to Dagger, MVP pattern, Espresso and more. This article focuses on introducing a powerful test coverage technique for ensuring that the app unit tests are strong enough.

Introduction to Mutation Testing

One of the golden standard test coverage techniques is mutation testing. The main idea of mutation testing is to perform byte code modifications (mutations) to your original app source code and then run your app unit tests to check if they are strong enough to fail as a result of these mutations.

The byte code changes in Mutation Testing are named as faults (or mutations). After applying the source code changes, if your tests fail then the mutation is killed, and if your tests pass then the mutation is lived.

So in summary, If your tests fail by source code mutations then this means that your tests are really strong, if your tests pass then this means you need to modify your tests to make them strong enough (The quality of your tests can be gauged from the percentage of mutations killed).

Why we need Mutation Testing

Mutation tests have an advantage over Traditional test coverage (i.e line, statement, branch etc) because it does not only measure which code is executed by your tests but also it checks that your tests are actually able to detect faults in the executed code. This gives a strong measurement to your developed unit tests.

PIT

PIT is one of the mutation testing systems for Java. It has several advantages:

  1. Fast to execute.
  2. Integrated with Java build tools (Ant, Maven and Gradle).
  3. Active.

More information about PIT can be found in:
http://pitest.org/

Android Integration

Unfortunately, the official PIT Gradle plugin does not work fine with Android Gradle projects:
https://github.com/szpak/gradle-pitest-plugin

However, thanks to one of the community member, this support is now possible using this fork:
https://github.com/koral–/gradle-pitest-plugin

Application Configuration

In order to configure Android PIT plugin with our Android Dagger app:
1. Add PIT plugin to your top-level build.gradle file:

classpath 'pl.droidsonroids.gradle:gradle-pitest-plugin:0.0.3'

2. Configure PIT plugin in your app build.gradle file:

pitest {
   targetClasses = ['com.test.xyz.daggersample1.ui.presenter.*']  /* specify target classes to be mutated */
   excludedClasses = ['**Factory*'] /* Exclude Dagger generated classes */
   pitestVersion = "1.1.0" 
   threads = 4 /* specify number of threads */
   outputFormats = ['XML', 'HTML'] /* specify output format */
}

Checking Results

After configuration, finally, we can execute PIT tests by running:

./gradlew pitest

You can pick the PIT tests report under build/reports/pitest.

The following figure shows the mutation testing report in our Dagger sample:
PIT Report

The following figure shows survived and killed mutants:
Mutants

FYI, in the mutation report, Light green shows line coverage, dark green shows mutation coverage, Light pink show lack of line coverage, and dark pink shows lack of mutation coverage.

Source code

App Source code is available in:
https://github.com/hazems/Dagger-Sample/tree/pit1/app/src/test/java/com/test/xyz/daggersample1/ui/presenter

Top-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/build.gradle

Module-level Gradle file PIT configuration:
https://github.com/hazems/Dagger-Sample/blob/pit1/app/build.gradle

In our next article, let’s see how to kill these survived mutants which makes our tests not strong enough!

[1] Previous articles:
http://www.technicaladvices.com/2016/03/05/an-introduction-to-dagger-2-android-di-part-1/
http://www.technicaladvices.com/2016/03/09/an-introduction-to-dagger-2-android-di-part-2/
http://www.technicaladvices.com/2016/04/07/dagger-2-mvp-and-unit-testing-android-di-part-3/

Back from JavaOne China 2013

July 26, 2013 in JavaOne, JavaScript, Unit Testing

JavaOne China

JavaOne China

I just get back from JavaOne China that was held in Shanghai 22-25 July 2013. JavaOne China is one of the biggest Java Conferences in Asia. The conference organization was good and there were a lot of attendees in the conference sessions.

JavaOne China

JavaOne China

I had the chance to present “Efficient JavaScript Unit Testing” in 23 July, the room of the session was full of attendees and I was really amazed by the energy, enthusiasm, and responsiveness of the attendees during the session.

Photo with Chinese Developers

Photo with Chinese Developers

The session went great and I was glad to give a free copy of my JavaScript Unit Testing book to one of the attendees who answered a JavaScript quiz at the end of the session.

I uploaded my session below:

Aside from the conference, Shanghai is a modern city that has many good places to visit, I had the chance to visit three places there:
1. Yuyuan Garden and Old Town Area:
Yuyuan Garden Photos

2. Jing Temple:
Jingle Temple Photos

3. Oriental Pearl TV Tower:
Oriental Pearl TV Tower Photos

“Efficient JavaScript unit testing” video session in CONFESS 2012

May 29, 2012 in JavaScript, JsTestDriver, Unit Testing, Web 2.0

This video provides a presentation about how to make efficient JavaScript unit testing. I presented it in the CON-FESS 2012 international conference which was held in Leogang, Austria. I wish it can be useful.

See the presentation with the slides on Parleys.com

See the presentation with the slides on Parleys.com

and I included below the presentation:

“Efficient JavaScript unit testing” presentation in CONFESS 2012

May 11, 2012 in JavaScript, JsTestDriver, Unit Testing

I just got back from the International technical conference CONFESS 2012 which was held in Leogang, Austria. The conference was very cool, the hotel and the city were fantastic, the quality of the sessions and the speakers were great.

Now, I got finally some free time to share with you my presentation about my efficient JavaScript unit testing. I hope it will be useful for you. Once I get the video session, I will definitely share it with you as well.

Have a nice time.

JsTestDriver integration with Hudson

May 5, 2012 in Continuous Integration, JsTestDriver

Abstract

JsTestDriver is one of the most efficient JavaScript unit testing tools. Integrating JsTestDriver with continuous integration tools is a mandatory step for ensuring the web application quality. In this article, I will illustrate how this sort of integration can be done between JsTestDriver and Hudson.

Installing Hudson

To install Hudson on your machine, you need to download it from http://hudson-ci.org/.

Run the following command to start Hudson:

java -jar hudson-2.2.0.war

You can start launching Hudson by typing the following address in your browser:

http://localhost:8080/

You should now be redirected to the Hudson home page.

Creating Hudson Job

First of all, you need to create a new Job in Hudson. You can do this by clicking on “New Job” and entering the Job name and selecting “Build a free-style software project” as shown in the figure below:

Configuring JsTestDriver with Hudson

Click the “Configure” link of the project. In the configuration page, enter the project description. In order to run the job periodically, you need to configure the “Build Triggers”. Check the “Build periodically” option and enter the cron-like expression. To run the job every 5 mins past the hour (for example), enter the following expression 5 * * * * as shown below in the figure:

In order to execute the JsTestDriver test command, you need to configure the “Build” section. There are many ways to do so, you can either add “Execute Shell”, “Invoke Ant or Maven2 or Maven3”, or “Execute Windows Batch command”. The latter case can be used when you are working on Windows environment. For the case of this example, I use the “Execute Windows Batch command” option and specify the path of the batch file as shown in the screenshot below:

The batch file has the following content:

java -jar JsTestDriver-1.3.4.b.jar 
      --config jsTestDriver.conf 
      --tests all 
      --testOutput c:/outputFolder 
      --reset

For the matter of simplicity, I assume that the JsTestDriver server is already started and some browsers are already captured. This step is not included in this post, however, This step also can be automated if required.

Testing the integration of JsTestDriver with Hudson

In order to test the integration of JsTestDriver with Hudson, this can be done by manually or automatically (as indicated in the “Build periodically” option). To test the integration manually, click on the “Build Now” link as shown in the screenshot below:

Click the “console output” link to check the output of the results:

setting runnermode QUIET
Firefox: Reset
...
Total 3 tests (Passed: 3; Fails: 0; Errors: 0) (3.00 ms)
  Firefox 12.0 Windows: Run 3 tests (Passed: 3; Fails: 0; Errors 0) (3.00 ms)

Notices

You may face this error while trying to run a new Job:

hudson java.io.IOException: Cannot run program "sh"

In order to fix this error, check this post:
http://www.technicaladvices.com/2012/05/04/hudson-java-io-ioexception-cannot-run-program-sh-resolution/.

More Information about JsTestDriver

Efficient JavaScript unit testing

Generating Test Reports using JsTestDriver

Running Jasmine on the top of the JsTestDriver test runner

Testing JavaScript code using Jasmine (Kickstart)

April 7, 2012 in JavaScript, Unit Testing, Web 2.0

Jasmine is one of the JavaScript unit testing frameworks. It has a nice advantage that its syntax is readable and a very human friendly. In this post, I will give you a kickstart to help you start working with Jasmine.

First of all, Let’s create a simple JavaScript object that we need to perform some unit testing on it:

/* Basics.js file */
function Basics() {
}

Basics.prototype.add = function(x, y) {
	return x + y;
};

As shown in the Basics.js file, Basics is a simple JavaScript object that contains one method that adds the x and y parameters and finally returns the result to the caller.

To start working with the Jasmine framework, download the framework from http://pivotal.github.com/jasmine/download.html. Make sure that you have the following folder structure after extracting the downloaded zip file:
jasmine structure

You are having three folders and one html file.
1. lib folder: contains the source files of the Jasmine framework.
2. src folder: contains the source JavaScript files that will be tested.
3. spec folder: contains the testing JavaScript files.
4. SpecRunner.html file: is used for running the JavaScript test suites.

Place the Basics.js in the src folder after making sure that you remove all the old contents of the src and the spec folders.

Now, Let’s write the test suite in the BasicsSpec.js file.

describe("Basics", function() {
  var basics = new Basics();  

  it("should be able to make correct addition", function() {
    expect(basics.add(3, 2)).toEqual(5);
  });

});

As we see here, the testing syntax is very human friendly, it means “describe Basics object which should be able to make correct addition”. The describe keyword represents a test suite that can contain one or more specs. Every spec is represented by the it keyword.

Inside the spec, there is an expectation that expects the basics.add(3, 2) is equal to 5. In Jasmine, the expectation is represented by the expect keyword and the equality checking is represented by a matcher called toEquals.

In Jasmine, there are many other built-in matchers; you can find all of them here: https://github.com/pivotal/jasmine/wiki/Matchers.

Finally, do not forget to remove the old source and spec JS includes from the SpecRunner.html and to add the new includes instead:

  <!-- include source files here... -->
  <script type="text/javascript" src="src/Basics.js"></script>  
  
  <!-- include spec files here... -->
  <script type="text/javascript" src="spec/BasicsSpec.js"></script>

After running the SpecRunner.html file, you will find the following screenshot telling you that the test spec runs successfully.

screenshot

I wish this kickstart can be useful for you. For your reference, download the kickstart example from here.

Generating Test Reports using JsTestDriver

March 10, 2012 in JavaScript, JsTestDriver, Unit Testing

JsTestDriver is one of the most powerful JavaScript unit testing tools. I wrote an article in developerWorks about efficient JavaScript unit testing using JsTestDriver, please read first it before reading this post.

In this post, I will show you how to generate test coverage reports using the JsTestDriver. First of all, Let’s understand what is meant by “Test Coverage”. “Test Coverage” is one of the measures of software testing. It describes how much the source code of the program is tested.

The coverage criteria can be on the function level (Function coverage), on the statement level (Statement coverage), and on the code branch level (Branch coverage).

JsTestDriver has a plugin that can generate code coverage for the JavaScript code. To configure this plugin, you should:

  • Download the “coverage.jar” from http://code.google.com/p/js-test-driver/downloads/list
  • Add the coverage plugin declaration to the configuration file (jsTestDriver.conf file) as follows:

    ...
    plugin: 
    - name: "coverage" 
      jar: "plugins/coverage.jar" 
      module: "com.google.jstestdriver.coverage.CoverageModule"
    
  • Specify the --testOutput [output_folder] flag in the test running command. This will get the test coverage files in the output_folder you specified.

Currently, The JsTestDriver generates the test coverage files in LCOV format. Unfortunately, JsTestDriver does not generate HTML reports directly as shown in the figure below.
Genarated LCOV files

To generate the HTML reports from the LCOV files, you can use the LCOV visualizer, check it here: http://ltp.sourceforge.net/coverage/lcov.php.

The LCOV visualizer tool is working on Linux Redhat environment. To generate the report from the tool, execute this command on the jsTestDriver.conf-coverage.dat file:

genhtml jsTestDriver.conf-coverage.dat

For more information about how to install the LCOV visualizer on Linux Redhat environment, and converting the LCOV files to HTML reports:
http://www.technicaladvices.com/2012/05/02/converting-the-jstestdriver-coverage-files-to-html-reports/.

Finally for your reference, You can download the project from here, and the LCOV test coverage files from here.

I wish that this tip can be useful.

Skip to toolbar