February 2, 2014

AES-256 Encryption with Java and JCEKS

Overview

Security has become a great topic of discussion in the last few years due to the recent releasing of documents from Edward Snowden and the explosion of hacking against online commerce stores like JC Penny, Sony and Target. While this post will not give you all of the tools to help prevent the use of illegally sourced data, this post will provide a starting point for building a set of tools and tactics that will help prevent the use of data by other parties.

This post will show how to adopt AES encryption for strings in a Java environment. It will talk about creating AES keys and storing AES keys in a JCEKS keystore format. A working example of the code in this blog is located at https://github.com/mike-ensor/aes-256-encryption-utility

It is recommended to read each section in order because each section builds off of the previous section, however, this you might want to just jump quickly jump to a particular section.

  • Setup - Setup and create keys with keytool
  • Encrypt - Encrypt messages using byte[] keys
  • Decrypt - Decrypt messages using same IV and key from encryption
  • Obtain Keys from Keystore - Obtain keys from keystore via an alias

What is JCEKS?

JCEKS stands for Java Cryptography Extension KeyStore and it is an alternative keystore format for the Java platform. Storing keys in a KeyStore can be a measure to prevent your encryption keys from being exposed. Java KeyStores securely contain individual certificates and keys that can be referenced by an alias for use in a Java program. Java KeyStores are often created using the "keytool" provided with the Java JDK.

NOTE: It is strongly recommended to create a complex passcode for KeyStores to keep the contents secure. The KeyStore is a file that is considered to be public, but it is advisable to not give easy access to the file.

Setup

All encryption is governed by laws of each country and often have restrictions on the strength of the encryption. One example is that in the United States, all encryption over 128-bit is restricted if the data is traveling outside of the boarder. By default, the Java JCE implements a strength policy to comply with these rules. If a stronger encryption is preferred, and adheres to the laws of the country, then the JCE needs to have access to the stronger encryption policy. Very plainly put, if you are planning on using AES 256-bit encryption, you must install the Unlimited Strength Jurisdiction Policy Files. Without the policies in place, 256-bit encryption is not possible.

Installation of JCE Unlimited Strength Policy

This post is focusing on the keys rather than the installation and setup of the JCE. The installation is rather simple with explicit instructions found here (NOTE: this is for JDK7, if using a different JDK, search for the appropriate JCE policy files).

Keystore Setup

When using the KeyTool manipulating a keystore is simple. Keystores must be created with a link to a new key or during an import of an existing keystore. In order to create a new key and keystore simply type:

keytool -genseckey -keystore aes-keystore.jck -storetype jceks -storepass mystorepass -keyalg AES -keysize 256 -alias jceksaes -keypass mykeypass 

Important Flags

In the example above here are the explanations for the keytool's parameters:

Keystore Parameters

genseckey
Generate SecretKey. This is the flag indicating the creation of a synchronous key which will become our AES key
keystore
Location of the keystore. If the keystore does not exist, the tool will create a new store. Paths can be relative or absolute but must be local
storetype
this is the type of store (JCE, PK12, JCEKS, etc). JCEKS is used to store symmetric keys (AES) not contained within a certificate.
storepass
password related to the keystore. Highly recommended to create a strong passphrase for the keystore

Key Parameters

keyalg
algorithm used to create the key (AES/DES/etc)
keysize
size of the key (128, 192, 256, etc)
alias
alias given to the newly created key in which to reference when using the key
keypass
password protecting the use of the key

Encrypt

As it pertains to data in Java and at the most basic level, encryption is an algorithmic process used to programmatically obfuscate data through a reversible process where both parties have information pertaining to the data and how the algorithm is used. In Java encryption, this involves the use of a Cipher. A Cipher object in the JCE is a generic entry point into the encryption provider typically selected by the algorithm. This example uses the default Java provider but would also work with Bouncy Castle.

Generating a Cipher object

Obtaining an instance of Cipher is rather easy and the same process is required for both encryption and decryption. (NOTE: Encryption and Decryption require the same algorithm but do not require the same object instance)

Cipher cipher = Cipher.getInstance("AES/CBC/PKCS5Padding");
Once we have an instance of the Cipher, we can encrypt and decrypt data according to the algorithm. Often the algorithm will require additional pieces of information in order to encrypt/decrypt data. In this example, we will need to pass the algorithm the bytes containing the key and an initial vector (explained below).

Initialization

In order to use the Cipher, we must first initialize the cipher. This step is necessary so we can provide additional information to the algorithm like the AES key and the Initial Vector (aka IV).

cipher.init(Cipher.ENCRYPT_MODE, secretKeySpecification, initialVector);

Parameters

The SecretKeySpecification is an object containing a reference to the bytes forming the AES key. The AES key is nothing more than a specific sized byte array (256-bit for AES 256 or 32 bytes) that is generated by the keytool (see above).

Alternative Parameteters

There are multiple methods to create keys such as a hash including a salt, username and password (or similar). This method would utilize a SHA1 hash of the concatenated strings, convert to bytes and then truncate result to the desired size. This post will not show the generation of a key using this method or the use of a PBE key method using a password and salt. The password and/or salt usage for the keys is handled by the keytool using the inputs during the creation of new keys.

Initialization Vector

The AES algorithm also requires a second parameter called the Initialiation Vector. The IV is used in the process to randomize the encrypted message and prevent the key from easy guessing. The IV is considered a publicly shared piece of information, but again, it is not recommended to openly share the information (for example, it wouldn't be wise to post it on your company's website). When encrypting a message, it is not uncommon to prepend the message with the IV since the IV will be a set/known size based on the algorithm. NOTE: the AES algorithm will output the same result if using the same IV, key and message. It is recommended that the IV be randomly created each time an encryption takes place.

With the newly initialized Cipher, encrypting a message is simple. Simply call:

byte[] encryptedMessageInBytes = Cipher.doFinal((message.getBytes("UTF-8"));
String base64EncodedEncryptedMsg = BaseEncoding.base64().encode(encryptedMessageInBytes);
String base32EncodedEncryptedMsg = BaseEncoding.base32().encode(encryptedMessageInBytes);

Encoding Results

Byte arrays are difficult to visualize since they often do not form characters in any charset. The best recommendation to solve this is to represent the bytes in HEX (base-16), Double HEX (base-32) or Base64 format. If the message will be passed via a URL or POST parameter, be sure to use a web-safe Base64 encoding. Google Guava library provides a excellent BaseEncoding utility. NOTE: Remember to decode the encoded message before decrypting.

Decrypt

Decrypting a message is almost a reverse of the encryption process with a few exceptions. Decryption requires a known initialization vector as a parameter unlike the encryption process generating a random IV.

Decryption

When decrypting, obtain a cipher object with the same process as the encryption method. The Cipher object will need to utilize the exact same algorithm including the method and padding selections. Once the code has obtained a reference to a Cipher object, the next step is to initialize the cipher for decryption and pass in a reference to a key and the initialization vector.

// key is the same byte[] key used in encryption
SecretKeySpec secretKeySpecification = new SecretKeySpec(key, "AES");
cipher.init(Cipher.DECRYPT_MODE, secretKeySpecification, initialVector);
NOTE: The key is stored in the keystore and obtained by the use of an alias. See below for details on obtaining keys from a keystore

Once the cipher has been provided the key, IV and initialized for decryption, the cipher is ready to perform the decryption.

byte[] encryptedTextBytes = BaseEncoding.base64().decode(message);
byte[] decryptedTextBytes = cipher.doFinal(encryptedTextBytes);
String origMessage = new String(decryptedTextBytes);

Strategies to keep IV

The IV used to encrypt the message is important to decrypting the message therefore the question is raised, how do they stay together. One solution is to Base Encode (see above) the IV and prepend it to the encrypted and encoded message: Base64UrlSafe(myIv) + delimiter + Base64UrlSafe(encryptedMessage). Other possible solutions might be contextual such as including an attribute in an XML file with the IV and one for the alias to the key used.

Obtain Key from Keystore

The beginning of this post has shown how easy it is to create new AES-256 keys that reference an alias inside of a keystore database. The post then continues on how to encrypt and decrypt a message given a key, but has yet shown how to obtain a reference to the key in a keystore.

Solution

// for clarity, ignoring exceptions and failures
InputStream keystoreStream = new FileInputStream(keystoreLocation);

KeyStore keystore = KeyStore.getInstance("JCEKS");
keystore.load(keystoreStream, keystorePass.toCharArray());

if (!keystore.containsAlias(alias)) {
    throw new RuntimeException("Alias for key not found");
}

Key key = keystore.getKey(alias, keyPass.toCharArray());

Parameters

keystoreLocation
String - Location to local keystore file location
keypass
String - Password used when creating or modifying the keystore file with keytool (see above)
alias
String - Alias used when creating new key with keytool (see above)

Conclusion

This post has shown how to encrypt and decrypt string based messages using the AES-256 encryption algorithm. The keys to encrypt and decrypt these messages are held inside of a JCEKS formatted KeyStore database created using the JDK provided "keytool" utility. The examples in this post should be considered a solid start to encrypting/decrypting symmetric keys such as AES. This should not be considered the only line of defense when encrypting messages, for example key rotation. Key rotation is a method to mitigate risks in the event of a data breach. If an intruder obtains data and manages to hack a single key, the data contained in multiple files should have used several keys to encrypt the data thus bringing down risk of a total exposure loss.

All of the examples in this blog post have been condensed into a simple tool allowing for the viewing of keys inside of a keystore, an operation that is not supported out of the box by the JDK keytool. Each aspect of the steps and topics outlined in this post are available at: https://github.com/mike-ensor/aes-256-encryption-utility. NOTE: The examples, sample code and any reference is to be used at the sole implementers risk and there is no implied warranty or liability, you assume all risks.

January 22, 2013

How to Publish Maven Site Docs to BitBucket or GitHub Pages

Introduction

In this post we will Utilize GitHub and/or BitBucket's static web page hosting capabilities to publish our project's Maven 3 Site Documentation. Each of the two SCM providers offer a slightly different solution to host static pages. The approach spelled out in this post would also be a viable solution to "backup" your site documentation in a supported SCM like Git or SVN. This solution does not directly cover site documentation deployment covered by the maven-site-plugin and the Wagon library (scp, WebDAV or FTP).

There is one main project hosted on GitHub that I have posted with the full solution. The project URL is https://github.com/mike-ensor/clickconcepts-master-pom/. The POM has been pushed to Maven Central and will continue to be updated and maintained.

<parent>
    <groupId>com.clickconcepts.project</groupId>
    <artifactId>master-site-pom</artifactId>
    <version>0.16</version>
</parent>

GitHub Pages

GitHub hosts static pages by using a special branch "gh-pages" available to each GitHub project. This special branch can host any HTML and local resources like JavaScript, images and CSS. There is no server side development.

To navigate to your static pages, the URL structure is as follows:

http://<some-username>.github.com/<project-name>
An example of the project I am using in this blog post: http://mike-ensor.github.com/clickconcepts-master-pom/ where the first bold URL segment is a username and the second bold URL segment is the project.

GitHub does allow you to create a base static hosted static site for your username by creating a repository with your username.github.com. The contents would be all of your HTML and associated static resources. This is not required to post documentation for your project, unlike the BitBucket solution.

There is a GitHub Site plugin that publishes site documentation via GitHub's object API but this is outside the scope of this blog post because it does not provide a single solution for GitHub and BitBucket projects using Maven 3.

BitBucket

BitBucket provides a similar service to GitHub in that it hosts static HTML pages and their associated static resources. However, there is one large difference in how those pages are stored. Unlike GitHub, BitBucket requires you to create a new repository with a name fitting the convention. The files will be located on the master branch and each project will need to be a directory off of the root.

mikeensor.bitbucket.org/
     /some-project
      +index.html
      +...
          /css
          /img
     /some-other-project
      +index.html
      +...
          /css
          /img
index.html
.git
.gitignore

The naming convention is as follows:

<username>.bitbucket.org
An example of a BitBucket static pages repository for me would be: http://mikeensor.bitbucket.org/. The structure does not require that you create an index.html page at the root of the project, but it would be advisable to avoid 404s.

Generating Site Documentation

Maven provides the ability to post documentation for your project by using the maven-site-plugin. This plugin is difficult to use due to the many configuration options that oftentimes are not well documented. There are many blog posts that can help you write your documentation including my post on maven site documentation. I did not mention how to use "xdoc", "apt" or other templating technologies to create documentation pages, but not to fear, I have provided this in my GitHub project.

Putting it all Together

The Maven SCM Publish plugin (http://maven.apache.org/plugins/maven-scm-publish-plugin/ publishes site documentation to a supported SCM. In our case, we are going to use Git through BitBucket or GitHub. Maven SCM Plugin does allow you to publish multi-module site documentation through the various properties, but the scope of this blog post is to cover single/mono module projects and the process is a bit painful.

Take a moment to look at the POM file located in the clickconcepts-master-pom project. This master POM is rather comprehensive and the site documentation is only one portion of the project, but we will focus on the site documentation. There are a few things to point out here, first, the scm-publish plugin and the idiosyncronies when implementing the plugin.

In order to create the site documentation, the "site" plugin must first be run. This is accomplished by running site:site. The plugin will generate the documentation into the "target/site" folder by default.

The SCM Publish Plugin, by default, looks for the site documents to be in "target/staging" and is controlled by the content parameter. As you can see, there is a mismatch between folders. NOTE: My first approach was to run the site:stage command which is supposed to put the site documents into the "target/staging" folder. This is not entirely correct, the site plugin combines with the distributionManagement.site.url property to stage the documents, but there is very strange behavior and it is not documented well.

In order to get the site plugin's site documents and the SCM Publish's location to match up, use the content property and set that to the location of the Site Plugin output (<siteOutputDirectory>).

If you are using GitHub, there is no modification to the siteOutputDirectory needed, however, if you are using BitBucket, you will need to modify the property to add in a directory layer into the site documentation generation (see above for differences between GitHub and BitBucket pages). The second property will tell the SCM Publish Plugin to look at the root "site" folder so that when the files are copied into the repository, the project folder will be the containing folder. The property will look like:

<siteOutputDirectory>${project.build.directory}/site/${project.artifactId}</siteOutputDirectory>
<scm-publish.siteDocOuputDirectory>${project.build.directory}/site</scm-publish.siteDocOuputDirectory>

Next we will take a look at the custom properties defined in the master POM and used by the SCM Publish Plugin above. Each project will need to define several properties to use the Master POM that are used within the plugins during the site publishing. Fill in the variables with your own settings.

BitBucket

<!-- Override Site Documentation SCM publishing parameters -->
<properties>
...
...
<scm-publish.scmBranch>master</scm-publish.scmBranch>
<scm-publish.pubScmUrl>scm:git:git@bitbucket.org:mikeensor/mikeensor.bitbucket.org.git</scm-publish.pubScmUrl>

<!-- Location of where "site" documentation is output; This is for BitBucket only!!! -->
<siteOutputDirectory>${project.build.directory}/site/${project.artifactId}</siteOutputDirectory>
<scm-publish.siteDocOuputDirectory>${project.build.directory}/site</scm-publish.siteDocOuputDirectory>

<!-- Overwrite from Parent Pom  -->
<changelog.fileUri>${changelog.bitbucket.fileUri}</changelog.fileUri>
<changelog.revision.fileUri>${changelog.revision.bitbucket.fileUri}</changelog.revision.fileUri>
...
...
</properties>

GitHub

<!-- Override Site Documentation SCM publishing parameters -->
<properties>
...
...
<scm-publish.scmBranch>gh-pages</scm-publish.scmBranch>
<scm-publish.pubScmUrl>scm:git:git@github.com:mikeensor/clickconcepts-master-pom.git</scm-publish.pubScmUrl>

<!-- Location of where "site" documentation is output; This is for 
<scm-publish.siteDocOuputDirectory>${project.build.directory}/site</scm-publish.siteDocOuputDirectory>

<!-- Overwrite from Parent Pom  -->
<changelog.fileUri>${changelog.github.fileUri}</changelog.fileUri>
<changelog.revision.fileUri>${changelog.revision.github.fileUri}</changelog.revision.fileUri>
...
...
</properties>
NOTE: changelog parameters are required to use the Master POM and are not directly related to publishing site docs to GitHub or BitBucket

How to Generate

If you are using the Master POM (or have abstracted out the Site Plugin and the SCM Plugin) then to generate and publish the documentation is simple.

mvn clean site:site scm-publish:publish-scm
mvn clean site:site scm-publish:publish-scm -Dscmpublish.dryRun=true

Gotchas

In the SCM Publish Plugin documentation's "tips" they recommend creating a location to place the repository so that the repo is not cloned each time. There is a risk here in that if there is a git repository already in the folder, the plugin will overwrite the repository with the new site documentation. This was discovered by publishing two different projects and having my root repository wiped out by documentation from the second project. There are ways to mitigate this by adding in another folder layer, but make sure you test often!

Another gotcha is to use the -Dscmpublish.dryRun=true to test out the site documentation process without making the SCM commit and push

Project and Documentation URLs

Here is a list of the fully working projects used to create this blog post:

November 19, 2012

How to test a Custom Exception using custom FEST assertions

Introduction

This is part three of my posts on assertions testing using Fest, JUnit and custom Exceptions. The first post was covering the basics of assertions, then followed up with testing custom Exceptions using JUnit and JUnit's ExpectedException class. At this point you should know that you have a custom Runtime Exception class and you would like to test it using the Fluent API provided by FEST.

Custom Assertions with Fest

This blog post will not go into the details on creating a custom assertion, but the solution posted in the Github project does contain a custom assertion. In addition, please refer to the official site docs.

Building off of the last post, you will see that ExpectedException is a great improvement over the @Test(expected) and Try/Catch techniques, however the ExpectedException object can still be improved by adding in a fluent-style API backed by Fest Assertion project. So, how do you do it? Let's get right to the solution!

Expected Exceptions with FEST and JUnit @Rule

Now that we have an understanding of FEST assertions, JUnit's @Rule functionality and ClickConcept's @ExpectedFailure we can combine the first two to provide fluent-style expected exception behavior while testing the assertion class using @ExpectedFailure annotations.

Testing your custom exception with FEST

Let's begin by creating a new @Rule object "ExpectedException" which extends TestRule. When creating the class, we will expose the construction through a simple factory method to return a new ExpectedException. The default factory will return a base implementation where the functionality is muted in all other cases where exceptions are not desired.

We can start out with the code first, but I will explain that in order to build your own custom Fluent API for FEST, you must re-create the API for the base exception assertion. The fluent API you create will be in addition to the FEST exception assertion class. Fluent API help was derived off of several blogs, but the most informative has been http://www.unquietcode.com/blog/2011/programming/using-generics-to-build-fluent-apis-in-java/.

NOTE: AbstractExpectedException encapsulates the base API for FEST's ExceptionAssertion. The code for this is found at the Github site: https://github.com/mike-ensor/fest-backed-expected-exception


public class ExpectedCustomException extends AbstractExpectedException<ExpectedCustomException> {

    private Integer code;

    public static ExpectedCustomException none() {
        return new ExpectedCustomException();
    }

    /**
     * Checks to see if the CustomException has the specified code
     *
     * @param code int
     * @return AbstractExpectedException
     */
    public AbstractExpectedException hasCode(int code) {
        // method telling class that a custom exception is being asked for
        markExpectedException();
        this.code = code;
        return this;
    }

    @Override
    protected void checkAssertions(Exception e) {
        // check parent's exceptions
        super.checkAssertions(e);

        if (getCode() != null) {
            // FEST Custom Assert object
            CustomExceptionAssert.assertThat(e).hasCode(code);
        }
    }

    private Integer getCode() {
        return code;
    }

}

Analysis

In this example, my CustomException has exposed a "code" to store when the exception was created. In order to test this my custom ExpectedException object must look for the proper code on the CustomException object, in a fluent manor.

Here is an example test case to explain how to use your new fluent API Custom Exception test. Take note of the third test case to see the Fluent API in use! (NOTE: Full test cases are available on my github account.

public class CustomExceptionTest {

    @Rule
    public ExpectedCustomException exception =
            ExpectedCustomException.none();

    @Rule
    public ExpectedTestFailureWatcher expectedTestFailureWatcher =
            ExpectedTestFailureWatcher.instance();

    @Test
    public void hasCode_worksAsExpected() {
        exception.hasCode(123);
        throw new CustomException("Message", 123);
    }

    @Test
    @ExpectedFailure
    public void getCode_fails() {
        exception.hasCode(456);
        throw new CustomException("Message", 123);
    }

    @Test
    @ExpectedFailure
    public void getMessageAndCode_codeFailsFirst() {
        exception.hasCode(456).hasMessage("Message");
        throw new CustomException("Message", 123);
    }

}

Summary

Thank you to those of you who have read through this little series covering assertions, how to test exceptions (both the exception flow and custom exceptions) and then on to testing your custom exceptions using FEST assertions. Please come back to my blog in the near future where I will have a REST API checklist to look over when architecting your next REST API.

Those who are reading this blog are most likely a small subset of the software development community, but if you are not, and you find the idea of a fluent-API really cool (as I do), please check out the FEST assertion library. If you are new to test driven development, please take up the practice and try applying to your code immediately. If all developers used TDD as a general practice, the level in quality would grow world wide!

September 25, 2012

Testing Custom Exceptions w/ JUnit's ExpectedException and @Rule


Exception Testing

Why test exception flows? Just like with all of your code, test coverage writes a contract between your code and the business functionality that the code is supposed to produce leaving you with a living documentation of the code along with the added ability to stress the functionality early and often. I won't go into the many benefits of testing instead I will focus on just Exception Testing.

There are many ways to test an exception flow thrown from a piece of code. Lets say that you have a guarded method that requires an argument to be not null. How would you test that condition? How do you keep JUnit from reporting a failure when the exception is thrown? This blog covers a few different methods culminating with JUnit's ExpectedException implemented with JUnit's @Rule functionality.


The "old" way

In a not so distant past the process to test an exception required a dense amount of boilerplate code in which you would start a try/catch block, report a failure if your code did not produce the expected behavior and then catch the exception looking for the specific type. Here is an example:

public class MyObjTest {

    @Test
    public void getNameWithNullValue() {

        try {
            MyObj obj = new MyObj();
            myObj.setName(null);
            
            fail("This should have thrown an exception");

        } catch (IllegalArgumentException e) {
            assertThat(e.getMessage().equals("Name must not be null"));
        }
    }
}

As you can see from this old example, many of the lines in the test case are just to support the lack of functionality present to specifically test exception handling. One good point to make for the try/catch method is the ability to test the specific message and any custom fields on the expected exception. We will explore this a bit further down with JUnit's ExpectedException and @Rule annotation.


JUnit adds expected exceptions

JUnit responded back to the users need for exception handling by adding a @Test annotation field "expected". The intention is that the entire test case will pass if the type of exception thrown matched the exception class present in the annotation.

public class MyObjTest {

    @Test(expected = IllegalArgumentException.class)
    public void getNameWithNullValue() {
        MyObj obj = new MyObj();
        myObj.setName(null);
    }
}

As you can see from the newer example, there is quite a bit less boiler plate code and the test is very concise, however, there are a few flaws. The main flaw is that the test condition is too broad. Suppose you have two variables in a signature and both cannot be null, then how do you know which variable the IllegalArgumentException was thrown for? What happens when you have extended a Throwable and need to check for the presence of a field? Keep these in mind as you read further, solutions will follow.


JUnit @Rule and ExpectedException

If you look at the previous example you might see that you are expecting an IllegalArgumentException to be thrown, but what if you have a custom exception? What if you want to make sure that the message contains a specific error code or message? This is where JUnit really excelled by providing a JUnit @Rule object specifically tailored to exception testing. If you are unfamiliar with JUnit @Rule, read the docs here.


ExpectedException

JUnit provides a JUnit class ExpectedException intended to be used as a @Rule. The ExpectedException allows for your test to declare that an exception is expected and gives you some basic built in functionality to clearly express the expected behavior. Unlike the @Test(expected) annotation feature, ExpectedException class allows you to test for specific error messages and custom fields via the Hamcrest matchers library.

An example of JUnit's ExpectedException

import org.junit.rules.ExpectedException;

public class MyObjTest {

    @Rule
    public ExpectedException thrown = ExpectedException.none();

    @Test
    public void getNameWithNullValue() {
        thrown.expect(IllegalArgumentException.class);
        thrown.expectMessage("Name must not be null");

        MyObj obj = new MyObj();
        obj.setName(null);
    }
}

As I eluded to above, the framework allows you to test for specific messages ensuring that the exception being thrown is the case that the test is specifically looking for. This is very helpful when the nullability of multiple arguments is in question.


Custom Fields

Arguably the most useful feature of the ExpectedException framework is the ability to use Hamcrest matchers to test your custom/extended exceptions. For example, you have a custom/extended exception that is to be thrown in a method and inside the exception has an "errorCode". How do you test that functionality without introducing the boiler plate code from the try/catch block listed above? How about a custom Matcher!

This code is available at: https://github.com/mike-ensor/custom-exception-testing


Solution: First the test case

import org.junit.rules.ExpectedException;

public class MyObjTest {

    @Rule
    public ExpectedException thrown = ExpectedException.none();

    @Test
    public void someMethodThatThrowsCustomException() {
        thrown.expect(CustomException.class);
        thrown.expect(CustomMatcher.hasCode("110501"));

        MyObj obj = new MyObj();
        obj.methodThatThrowsCustomException();
    }
}

Solution: Custom matcher

import com.thepixlounge.exceptions.CustomException;
import org.hamcrest.Description;
import org.hamcrest.TypeSafeMatcher;

public class CustomMatcher extends TypeSafeMatcher<CustomException> {

    public static BusinessMatcher hasCode(String item) {
        return new BusinessMatcher(item);
    }

    private String foundErrorCode;
    private final String expectedErrorCode;

    private CustomMatcher(String expectedErrorCode) {
        this.expectedErrorCode = expectedErrorCode;
    }

    @Override
    protected boolean matchesSafely(final CustomException exception) {
        foundErrorCode = exception.getErrorCode();
        return foundErrorCode.equalsIgnoreCase(expectedErrorCode);
    }

    @Override
    public void describeTo(Description description) {
        description.appendValue(foundErrorCode)
                .appendText(" was not found instead of ")
                .appendValue(expectedErrorCode);
    }
}

NOTE: Please visit https://github.com/mike-ensor/custom-exception-testing to get a copy of a working Hamcrest Matcher, JUnit @Rule and ExpectedException.

And there you have it, a quick overview of different ways to test Exceptions thrown by your code along with the ability to test for specific messages and fields from within custom exception classes. Please be specific with your test cases and try to target the exact case you have setup for your test, remember, tests can save you from introducing side-effect bugs!

September 18, 2012

Brief Overview of Java Assertions

What are asserts?

An assertion is a predicate (a true–false statement) placed in a program to indicate that the developer thinks that the predicate is always true at that place. [wikipedia]

Traditional asserts

Traditional testing frameworks started with the built in keyword assert.

An example of the keyword assert:

assert <condition> value

Assert has a few drawbacks including stopping the test execution and lengthy and hard to describe assert statements.

Second generation assertions

Along comes JUnit's assert framework. Built on top of the assert keyword, JUnit provided developers the ability to be more descriptive about the testing statements.

An example of JUnit's asserts:
// asserts that the condition must be true
assertTrue("This should be true", "abc".equalsIgnoreCase("ABC"));
// asserts that the object must not be null
assertNotNull(new MyObject());
//...
assertFalse(false == true);
//
assertNull(null);
// etc...

While there are some improvements on readability and usability to the basic assert keyword provided by JUnit, they share some of the same drawbacks in that many developers just use the "assertTrue()", "assertEquals()" and "assertFalse()" methods still providing a very cryptic assertion statement.

Third generation assertions

In an effort to guide developers into writing test assertions that are more readable and usable the Hamcrest library was created which switched the philosophy from many assert functions to just one basic function. The fundamental thought is that the assert is the same but the conditions will change. Hamcrest was built using the concepts of BDD (behavior driven design) where the test assertion is closer to that of a sentence.

An example of hamcrest assertions

@Test
public void showoffSomeHamcrestAssertsAndMatchers() {
    // asserts that string "abc" is "ABC" ignoring case
    assertThat("abc", is(equalToIgnoringCase("ABC")));
    assertThat(myObject.getFirst(), is("Mike!"));
    assertThat(myObject.getAddress(), is(notNullValue));
}

Hamcrest is a great improvement on top of the JUnit framework providing a flexible and readable testing platform. Hamcrest + JUnit is a comprehensive testing framework and when combined with Mockito (or other mocking framework) can provide a very descriptive and thorough unit testing solution. One of the drawbacks to using Hamcrest is that while descriptive, multiple assertions must be made to ensure that a test case has been covered. Many TDD purest agree that a test case should contain one and only one assertion, but how is this possible with a complex object? (purest will say refactoring, but oftentimes this is not feasible)

Fourth generation frameworks

And finally we come to the present with the latest assertion frameworks. Fest Assertion framework takes off where Hamcrest stopped providing a fluent style assertion framework giving the developer the ability to test objects with a single assertion.

An example of Fest assertions

@Test
public void getAddressOnStreet() {
    List<Address> addresses = addressDAO.liveOnStreet("main");
    assertThat(addresses).hasSize(10).contains(address1, address2);
    assertThat(stringObj).hasSize(7).isEqualToIgnoringCase("abcdefg");
}

As you can see, FEST exceptions provide a cleaner approach to assertions. FEST is very extensible and easy to use.

Conclusion

Assertions are a key tool in a professional developer's toolbox in which to stress test their code. This post is a part in a series with the culmination being a fluent style ExpectedException mechanism, backed by Fest, in which to add better clarity to your test cases when exceptions are being introduced. Feel free to read the lead up to this post; Allowing known failing JUnit Tests to pass test cases.

NOTE: This blog is providing an overview on assertion frameworks it should be noted that there are various other second and third generation testing frameworks that strive to provide better clarity and usability for testing including notables such as TestNG and JTest

September 9, 2012

Allowing JUnit Tests to Pass Test Case on Failures


Why create a mechanism to expect a test failure?

There comes a time when one would want and expect a JUnit @Test case fail. Though this is pretty rare, it happens. I had the need to detect when a JUnit Test fails and then, if expected, to pass instead of fail. The specific case was that I was testing a piece of code that could throw an Assert error inside of a call of the object. The code was written to be an enhancement to the popular new Fest Assertions framework, so in order to test the functionality, one would expect test cases to fail on purpose.


A Solution

Jump to the Code
One possible solution is to utilize the functionality provided by a JUnit @Rule in conjunction with a custom marker in the form of an annotation.


Why use a @Rule?

@Rule objects provide an AOP-like interface to a test class and each test cases. Rules are reset prior to each test case being run and they expose the workings of the test case in the style of an @Around AspectJ advice would.

Required code elements

  • @Rule object to check the status of each @Test case
  • @ExpectedFailure custom marker annotation
  • Test cases proving code works!
  • Optional specific exception to be thrown if annotated test case does not fail

NOTE: working code is available on my github page and has been added to Maven Central. Feel free to Fork the project and submit a pull request

Maven Usage

<dependency>
    <groupId>com.clickconcepts.junit</groupId>
    <artifactId>expected-failure</artifactId>
    <version>0.0.9</version>
</dependency>

Example Usage

In this example, the "exception" object is a Fest assertion enhanced ExpectedException (look for my next post to expose this functionality). The expected exception will make assertions and in order to test those, the test case must be marked as @ExpectedFailure

public class ExceptionAssertTest {

    @Rule
    public ExpectedException exception = ExpectedException.none();

    @Rule
    public ExpectedTestFailureWatcher watcher = ExpectedTestFailureWatcher.instance();

    @Test
    @ExpectedFailure("The matcher should fail becasue exception is not a SimpleException")
    public void assertSimpleExceptionAssert_exceptionIsOfType() {
        // expected exception will be of type "SimpleException"
        exception.instanceOf(SimpleException.class);
        // throw something other than SimpleException...expect failure
        throw new RuntimeException("this is an exception");
    }
}

Implementation of Solution

Reminder, the latest code is available on my github page.

@Rule code (ExpectedTestFailureWatcher.java)

import org.junit.rules.TestRule;
import org.junit.runner.Description;
import org.junit.runners.model.Statement;
// YEAH Guava!!
import static com.google.common.base.Strings.isNullOrEmpty;

public class ExpectedTestFailureWatcher implements TestRule {

    /**
     * Static factory to an instance of this watcher
     *
     * @return New instance of this watcher
     */
    public static ExpectedTestFailureWatcher instance() {
        return new ExpectedTestFailureWatcher();
    }

    @Override
    public Statement apply(final Statement base, final Description description) {
        return new Statement() {
            @Override
            public void evaluate() throws Throwable {
                boolean expectedToFail = description.getAnnotation(ExpectedFailure.class) != null;
                boolean failed = false;
                try {
                    // allow test case to execute
                    base.evaluate();
                } catch (Throwable exception) {
                    failed = true;
                    if (!expectedToFail) {
                        throw exception; // did not expect to fail and failed...fail
                    }
                }
                // placed outside of catch
                if (expectedToFail && !failed) {
                    throw new ExpectedTestFailureException(getUnFulfilledFailedMessage(description));
                }
            }

            /**
             * Extracts detailed message about why test failed
             * @param description
             * @return
             */
            private String getUnFulfilledFailedMessage(Description description) {
                String reason = null;
                if (description.getAnnotation(ExpectedFailure.class) != null) {
                    reason = description.getAnnotation(ExpectedFailure.class).reason();
                }
                if (isNullOrEmpty(reason)) {
                    reason = "Should have failed but didn't";
                }
                return reason;
            }
        };
    }
}

@ExpectedFailure custom annotation (ExpectedFailure.java)

import java.lang.annotation.*;

/**
 * Initially this is just a marker annotation to be used by a JUnit4 Test case in conjunction
 * with ExpectedTestFailure @Rule to indicate that a test is supposed to be failing
 */
@Documented
@Retention(RetentionPolicy.RUNTIME)
@Target(value = ElementType.METHOD)
public @interface ExpectedFailure {
    // TODO: enhance by adding specific information about what type of failure expected
    //Class assertType() default Throwable.class;

    /**
     * Text based reason for marking test as ExpectedFailure
     * @return String
     */
    String reason() default "";
}

Custom Exception (Optional, you can easily just throw RuntimeException or existing custom exception)

public class ExpectedTestFailureException extends Throwable {
    public ExpectedTestFailureException(String message) {
        super(message);
    }
}

Can't one exploit the ability to mark a failure as expected?

With great power comes great responsibility, it is advised that you do not mark a test as being @ExpectedFailure if you do not understand exactly why the test if failing. It is recommended that this testing method be implemented with care. DO NOT use the @ExpectedFailure annotation as an alternative to @Ignore

Possible future enhancements could include ways to specify the specific assertion or the specific message asserted during the test case execution.

Known issues

In this current state, the @ExpectedFailure annotation can cover up additional assertions and until the future enhancements have been put into place, it is advised to use this methodology wisely.

March 19, 2012

Maven 3 Site Docs Part 1: Basic Site Documentation

What is documentation?

Documentation
The process of documenting knowledge

Documentation for software developers are documents that describe how to run or use the artifact/project. Typically documentation provides any exposed settings, their uses and default values. In addition information about integration, entry and exit points such as JMS queues, JMX MBeans, DataSource objects and/or JNDI variables in order to run and use the artifact. There are many programs and standards to produce this level of documentation including MS word, Sharepoint, doxygen, Latex, Wiki, markdown, textile, etc.

Outside of artifact specific documentation team leads, architects and project managers usually ask for JavaDocs to be included. JavaDocs, if combined with Maven provide more documentation for your IDE and better information for the consumers of your artifact. Other important documentation includes testing reports, test coverage, static analysis tools, dependencies of your artifact, change log and issue management such as Jira or Bugzilla.

Why create docs?

The best reason to care about documentation is to provide information for external developers and team members outside of the project's development environment. Documentation can be utilized inside of IDEs as well and face it, how many times do you wish you had sources and/or JavaDocs for a dependency when you were programming or to send back an email with the site docs so someone can be self-served rather than take your time.

Problems with traditional documentation

The largest complaint with traditional documentation is that it becomes stale over time. Time is never allocated in a project&dagger to provide or update documentation and stale docs can lead a developer or team down many wrong paths leading to hours or days of misinformation lead rabbit holes.

† Agile, SCRUM and Lean processes should allow time to be built-in to handle technical debt like documentation, even thought it is called the "necessary evil". Read more here

Secondly, developers are the ones who are askedforced to maintain the documentation and typically developers are not great in the writing department and unless segments of the documentation process are automated, developers are people and we forget.

Last major problem with traditional documentation is that it is typically not automated and is an extra step during the build/release cycle. Any process in the build cycle that is not automated and tested will break down at some point. Humans are great at abstract thinking, not repetitive processes.

Solution to the problems

As stated above, the main problems are developers maintaining all documentation and documentation that is either built outside of the immediate environment or is not automated. This is where Maven's site documentation comes in. Maven was built with the understanding that documentation needs to be automated and has been progressively improved throughout updates to the maven project.

The most common case for Maven site docs is to deploy generated documentation to an external web server via SCP, FTP, WebDAV or SFTP. The site plugin takes use of Apache's Wagon library for secure transfer of docs. Most of the time the site docs are generated during the release† or deploy phases and generate on the verify phase.

† Release isn't a phase in maven, it is a plugin, but it is considered a best practice when publishing versioned artifacts to a maven repository

Maven 3 vs Maven 2

Maven decided to change the way that reports were collected and generated in between the Maven 2 and Maven 3 releases. In Maven 2, the POM would have a section called >reports< where plugins that handled ReportSet would be handled. In Maven 3, there is now a plugin that handles and collates the ReportSets instead of Maven handling it. The plugin is called maven-project-info-reports-plugin and documentation can be found here.

Sources for this series of blog post

I have provided a link to a fully functional multi-module maven project that publishes to both http://mike-ensor.github.com/multi-module-site-doc-project/index.html and locally to ${env.HOME}/sitedocs/${project.artifactId}.

https://github.com/mike-ensor/multi-module-site-doc-project.

Maven 3 Site Docs

There are two important elements with respect to site docs for Maven. Plugins, which generate ReportSet elements and Site Doc/Descriptors.

Plugins

Plugins are an important part of the documentation process and will typically run throughout the maven build lifecycle generating artifacts for the plugin phase that generates the ReportSet. A good example would be the maven-checkstyle-plugin. The plugin is typically bound to the process-resources or packakge phases so that developers know they have a checkstyle violation early in the build lifecycle. The plugin generates a checkstyle.xml document that is then used in the site phase by the maven-project-info-reports-plugin to generate a checkstyle report page.

Maven Site Plugin

The maven-site-plugin is the main plugin that is used to crate all site documentation. Each plugin that produces and/or consumes a ReportSet will need to be configured inside of the <reportPlugins> section. NOTE: This is where the Wagon plugin is set as a dependency. The Site plugin will use the <distributionManagement><site> line to determine where the site docs are to be published (SCM, WebDAV, SFTP, etc)

<plugin>
 <artifactId>maven-site-plugin</artifactId>
 <version>${plugin.maven-site.version}</version>
 <dependencies>
  <dependency><!-- add support for ssh/scp -->
   <groupId>org.apache.maven.wagon</groupId>
   <artifactId>wagon-ssh</artifactId>
   <version>${plugin.wagon-ssh.version}</version>
  </dependency>
 </dependencies>
 <configuration>
  <attach>true</attach>
  <reportPlugins>
   <plugin>
   ...
   ... Plugins go here
   ....
   </plugin>
  </reportPlugins>
  <attach>true</attach>
 </configuration>
</plugin>

Maven Project Info Report Plugin

The maven-project-info-report-plugin is the main plugin used by the maven-site-plugin that Maven 3 utilizes to collate and produce the common site reporting docs for the artifact

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-project-info-reports-plugin</artifactId>
    <configuration>
        <dependencyDetailsEnabled>false</dependencyDetailsEnabled>
        <dependencyLocationsEnabled>false</dependencyLocationsEnabled>
    </configuration>
    <!-- simpler configuration without
        reportSets available for usual cases -->
    <!-- distribution-management,
        index, dependencies, help,
        issue-tracking, plugins,
        cim, license, dependency-management,
        mailing-list, project-team,
        dependency-convergence,
        scm, plugin-management,
        modules, summary -->
    <reports>
        <report>summary</report>
        <report>dependencies</report>
        <report>project-team</report>
        <report>issue-tracking</report>
        <report>scm</report>
        <report>cim</report>
        <report>modules</report>
        <report>plugins</report>
    </reports>
</plugin>

Take a quick look at the types of reports that are automatically generated using the maven-project-info-reports-plugin. Many of these automatically generated reports provide additional information on top of the redundant information provided by the below listed plugins.

List of automatically provided reports
  • distribution-management
  • index
  • dependencies
  • help
  • issue-tracking
  • plugins
  • cim
  • license
  • dependency-management
  • mailing-list
  • project-team
  • dependency-convergence
  • scm
  • plugin-management
  • modules
  • summary

Common Plugins

Checkstyle - maven-checkstyle-plugin – Provides static analysis on source files – NOTE: Checkstyle needs to be run during build AND report phases

Example report plugin

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-checkstyle-plugin</artifactId>
 <configuration>
  <skip>${checkstyle.skip}</skip>
  <configLocation>${checkstyle.configUrl}</configLocation>
  <failsOnError>false</failsOnError>
  <enableRulesSummary>true</enableRulesSummary>
  <includeTestSourceDirectory>true</includeTestSourceDirectory>
 </configuration>
</plugin>

NOTE: A best practice is to provide a flag to enable/disable the plugin on a project-by-project basis. Also, notice the failsOnError, this will ensure that the plugin will produce an artifact on error and present that to the user.

FindBugs - maven-findbugs-plugin – Provides static analysis on source files – Plugin does not need to be run outside of report phase

Example report plugin

<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>findbugs-maven-plugin</artifactId>
 <configuration>
  <skip>${findbugs.skip}</skip>
  <xmlOutput>true</xmlOutput>
 </configuration>
</plugin>

NOTE: A best practice is to provide a flag to enable/disable the plugin on a project-by-project basis. Also, notice the failsOnError, this will ensure that the plugin will produce an artifact on error and present that to the user.

Cobertura - maven-cobertura-plugin – Provides code coverage during Unit Test (test) phase. NOTE: Must be run during build AND during reporting phases

Example report plugin

<plugin>
 <groupId>org.codehaus.mojo</groupId>
 <artifactId>cobertura-maven-plugin</artifactId>
 <configuration>
  <skip>${cobertura.skip}</skip>
 </configuration>
</plugin>

NOTE: A best practice is to provide a flag to enable/disable the plugin on a project-by-project basis. Also, notice the failsOnError, this will ensure that the plugin will produce an artifact on error and present that to the user.

PMD and CPD - maven-pmd-plugin – Provides static analysis (PMD) and code duplication/cut-copy-paste detection for artifact. Read more about the PMD plugin here.

Example report plugin

<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-pmd-plugin</artifactId>
 <configuration>
  <skip>${pmd.skip}</skip>
  <targetJdk>${maven.compiler.source}</targetJdk>
  <sourceEncoding>${project.build.sourceEncoding}</sourceEncoding>
  <minimumTokens>100</minimumTokens>
  <rulesets>
   <ruleset>${maven-pmd-plugin.configLocation}</ruleset>
  </rulesets>
  <failOnViolation>false</failOnViolation>
 </configuration>
</plugin>

Surefire and Failsafe - maven-surefire-report-plugin – Creates reports based on tests cases for unit tests and optionally integration tests. NOTE, This is a different plugin than the maven-surefire-plugin, this is the report plugin that operates on the ResultSet for both Surefire and Failsafe. Both Surefire and Failsafe must be run during the build for this plugin to generate reports.

Example report plugin

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-report-plugin</artifactId>
    <reportSets>
        <reportSet>
            <id>integration-tests</id>
            <reports>
                <report>report-only</report>
                <report>failsafe-report-only</report>
            </reports>
        </reportSet>
    </reportSets>
</plugin>

JavaDoc - maven-javadoc-plugin – Bundles JavaDocs into the reporting structure. NOTE, this does not need to be run during the standard build, only during the reporting phase

Example report plugin

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-javadoc-plugin</artifactId>
    <configuration>
        <skip>${javadocs.skip}</skip>
        <failOnError>false</failOnError>
    </configuration>
</plugin>

JXR - maven-jx-plugin – Builds cross-link information for classes. Checkstyle, PMD, FindBugs, JavaDocs and various other plugins can utilize the JXR configuration. NOTE, this plugin must be run at an early place in the Maven lifecycle (suggest process-resources) in addition to the sitedoc plugin section.

Example report plugin

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-jxr-plugin</artifactId>
    <configuration>
        <aggregate>true</aggregate>
    </configuration>
</plugin>

Useful Plugins

Other handy plugins to use for site documentation include:

End of Part 1

The next post will cover the developer portion of the documentation; more specifically covering XDoc and APT. The site documentation is robust and the next post will show how to program some of the XDoc portions of the docs by leveraging Velocity templates.

The last part of this blog post will cover multi-module projects. In addition, that post will show how to deploy site-docs to a Github Pages so you can have site docs at http://<username>.github.com/<your-project>.