December 15, 2011

How to setup several GitHub accounts on multiple machines with separate SSH keys

Problem

So you've got a personal GitHub account, one for one client, one for another and you want to keep them all separate. How do you setup your box to clone private repositories based on which account you're using? Here's the simple guide on how to do this

Assumptions

I have set this up on Linux (Ubuntu 11.10) and Windows, but only via the Git Bash and/or Cygwin. This post will not cover how to setup and where to store SSH keys for Windows or Mac, though Mac should be fairly similar to Linux.

Setup first SSH key

I'm not going to go into depth on how to setup SSH keys, you can read more about that here. However, I will show the steps I used to get there. For first-account: (please replace "first-account" with your GitHub account)
  1. Change to your home .ssh folder
    cd ~/.ssh
    
  2. Setup your RSA key
    ssh-keygen -t rsa -f ~/.ssh/first-account-rsa -C "first-account@example.com"
    
  3. After following the directions, you should get something that looks like:
    The key fingerprint is:
    00:ff:00:cc:13:00:bb:ll:aa:hh
    
    The key's randomart image is:
    +--[ RSA 2048]----+
    |     .+   +      |
    |       = o P .   |
    |        = * *    |
    |       o = +     |
    |      o S .      |
    |     o o =       |
    |      o . E      |
    |                 |
    |                 |
    +-----------------+
    
    
  4. Open your GitHub.com account setup page, click on the "SSH Public Keys" tab.
  5. Create a new SSH key and copy the contents of ~/.ssh/first-account-rsa.pub and place into the new text field. Name it something unique so you can remember which machine you're using for your account.

    If you plan on each of your accounts needing access, open each of those accounts and add your SSH key to each account. This isn't very common, so only do this if you need to.
  6. To test, type in:
    ssh -t git@github.com
    

    As long as you do not see "access denied", you're good. You might see "failed on channel 0", this is ok.

Adding a second account

Now you should have your first account working, let's setup and add the second account.

  1. Repeat process 1-5 and stop (using the next account)
  2. Create a file named "config" in your ~/.ssh folder
    echo '' > ~/.ssh/config
    
    –or–
    touch ~/.ssh/config
    
  3. Use your favorite editor to add the configuration information for the SSH client to determine which RSA key to use:
    # first-account GitHub account
    Host github.com
    HostName github.com
    User git
    IdentityFile ~/.ssh/first-account-rsa # Note, this is the private key matching the .pub
    
    # second-account
    Host github-second-account  # NOTE: The host is NOT github.com, more on this further down
    HostName github.com
    User git
    IdentityFile ~/.ssh/second-account-rsa # Private key for the second account
    
    # repeat for the third and so on
    
  4. Save the file and your configuration is done

Let's look at the config file in a bit more detail. The first account is the base github.com host name, and you typically see a git URL as git@github.com:user/project.git. With your first account, you will leave this the domain the same**.

NOTE: This doesn't stop you from modifying your ~/.ssh/config file and instead of using "github.com" as your default host, setting the value to "github-first-account".

Now, for your second account, you will need to modify the URL you are using to clone the repository. This is done by changing the domain from "github.com" to "github-second-account". For example: git@github-second-account:user/project.git

Summary

You should now have the setup to clone and modify code for multiple accounts on the same machine. Repeat this process for each machine you would like to make available to GitHub.

December 3, 2011

Spring 3.1 + Cloud Foundry + Local Development

This post will help you build a Spring 3.1 web application using MongoDB on Cloud Foundry.  In addition to pushing to Cloud Foundry, you will also be able to develop in your local environment with a MongoDB instance.

Goals
The goals for this blog posting will be to build the application locally, then publish to your local Cloud Foundry instance.  We will utilize the Cloud Foundry runtime, and the new Spring Profiles

Setup
  1. Create an account with Cloud Foundry (https://www.cloudfoundry.com/micro)
  2. Follow instructions to setup your own Micro Cloud
    1. I use VMWare's player
    2. Verify "vmc info" that the micro cloud console matches
  3. Download MongoDB (at least version 2.0)
  4. Install and be familiar with Maven 3 (http://maven.apache.org)
  5. Familiarize yourself with Spring 3.1, Spring Data and Spring MongoDB
  6. Clone or download the source (https://github.com/mike-ensor/first-cloud-app)
  7. Run the app locally with:
    mvn clean package cargo:run -DskipTests
  8. Go to http://localhost:8080/home

Profiles
New in Spring 3.1 are the environment profiles which allow a developer to activate groups of beans based on an environment parameter.  There are several "gotchas" that I've discovered, one being an undocumented ordering for beans using profiles.

Take a look at data-services.xml.  Notice how the MongoTemplate is defined before the .  This is against my intuition because the MongoTemplate takes a reference to the MongoFactory object, which is defined below the MongoTemplate definition.

The second "gotcha" came from when and where to set the parameter to enable Spring's profiles.  The documentation and blogs do not explicitly mention that the developer must specify the profile that is active.  It was implied by the documentation that "default" was active by default, but this is not true.  In order for the default profile to be active, I added it as a system property in my cargo settings.  (as long as it is a system environment property, feel free to set it anywhere or any way you'd like).  Take a look at the pom.xml file around line 40 for the local Maven property and then around line 253 for the environment variable to be set.

Local development vs. Cloud Development
One of the main goals I had for interacting with Cloud Foundry is that I wanted a local development environment to speed up and ease development and reduce complexity with debugging.  Notice that in data-services.xml there is a "cloud" profile and a "default".  The point of the "default" profile is to have beans that are constructed when on a local environment.  You can see that there are two definitions of MongoFactory, one using Spring Data MongoDB's XML namespace and one using CloudFoundry Runtime's namespace.  I am not going to cover why these work the way they do, so if you'd like information, refer to http://blog.springsource.org/2011/04/12/cloud-foundry-for-spring-developers/ and http://blog.springsource.org/2011/11/09/using-cloud-foundry-services-with-spring-applications-part-3-the-cloud-namespace/

Pushing to Cloud Foundry
Now that you have a local running instance of the webapp, you will notice that the artifact is called "first-cloud-app.war", which you can find in the "/target" folder.  This is a problem when pushing to the Cloud Foundry instance since the name cannot contain any non-alpha characters.  Cloud Foundry's vmc tool is built from the VCAP open source project that is responsible for the open source PaaS services.  Another PaaS service includes App Fog, which allows you to basically use the same commands, but replace "vmc" for "af".  Both services fall victim to the naming problem.

In order to get around the naming problem, I have created a Maven profile cloud that builds the WAR artifact as "mikeensor.war".  Please change this to match your application's name since you won't have the user/password to publish (or the DNS) to publish to my micro instance.  The name will need to fit into the URL pattern http://<applicationname>..cloudfoundry.me.

To publish to your local cloud foundry micro instance, goto the root folder and type the following.  (this is assuming your micro instance is running and there are no "red" errors.

mvn clean package -Pcloud
vmc push <application name> -path target/

(if you have already pushed before, you will need to type:
vmc update <application name> -path target/

Note: It is possible to use the Maven Plugin for Cloud Foundry, however, I have still not been able to get it to work without changing the name of the artifact.

Enabling and connecting to services
You must create a service(s) so that your application can bind to the data source.  The VCAP (vmc) application handles how the configuration works when loading your application into Cloud Foundry.  It does this via an environment variable that is consumed in the namespaced configuration element.

In my example, I created a MongoDB service by typing:
vmc create-service mongodb --name <what you want to call your instance>

I named mine "second" (because I had created a first) and you will see that in data-services.xml the cloud XML configuration refers to the name of the service.
 
Note that if you have multiple MongoDB instances, you will need to do some Spring configuration (@Qualifier) when you want to use different instances.  This is not covered by this blog posting.

Now you will need to "bind" the service to your application.  This is done by typing:
vmc bind-service <name above> <application name>


Testing it out
You should be able to goto http://..cloudfoundry.me/home (example: http://mikeensor.mikeensor.cloudfoundry.me/home)

Congratulations!  You should have not only successfully deployed to Cloud Foundry (micro instance), bound to a MongoDB instance, but should be able to run in your local environment too!  As I get time, I will try to add in more detailed features such as multiple types of storage and post other "gotchas" as I find them.

August 2, 2011

Maven + JavaScript Unit Test: Running in a Continuous Integration Environment

JavaScript Unit Testing Continued

So you're still interested in unit testing JavaScript (good). This post is an extension of my much more indepth first posting on how to unit test JavaScript using JS Test Driver. Please check it out here.

Recap Last Posting

In the last posting we successfully unit tested JavaScript using Maven and JsTest Driver. This allowes us to test JavaScript when on an environment that has a modern browser installed and can be run.

Problem with typical CI environments

So what happens when the test are passing on your local box, but you go to check in your code and the Continuous Integration (CI) server pukes on the new tests becasue there is no "screen" to run chrome or firefox? As of this posting, none of the top-tier browsers have a "headless" or an in-memory only browser window. There are alternatives to running JavaScript in a browser, such as rhino.js, env.js or HtmlUnit, however, these are just ports of browsers and the JavaScript and DOM representation are not 100% accurate which can lead to problems with your code when rendered in a client's browser.

Approach

What we need to do is to run JSTestDriver's browser in a Virtual X Framebuffer (Xvfb) which is possible on nearly all Linux based systems. The example below uses a Solaris version of Linux, however, Debian and RedHat linux distrubutions come with the simplified bash script to easily run an appliation in a virtual framebuffer. This solution was derived from one posted solution on the JS Test Driver wiki (http://code.google.com/p/js-test-driver/wiki/ContinuousBuild. The given example is also a full working example that is in use at my current client.

Here is the quick list of what we will accomplish. Note, several of these steps are discussed in depth in the previous post and are not covered in depth here.
  1. Create a profile to run Js Unit-Tests
  2. Copy JsTestDriver library to a known location for Maven to use
  3. Copy JavaScript main and test files to known locations
  4. Use ANT to start JsTestDriver and pipe the screen into xvfb

Here is a sample profile to use. You will need to adjust the properties at the top of the profile to match your system.
<profile>
    <!-- Runs JS Unit Test inside of Xvfb -->
    <id>ci-jstests</id>
    <properties>
        <firefox.location>/opt/swf/bin/firefox</firefox.location>
        <js-test-driver.version>1.3.2</js-test-driver.version>
        <xvfb.location>/opt/X11R6/xvfb-run</xvfb.location>
    </properties>
    <build>
        <plugins>
            <!-- Copy JS Test Driver JAR to target folder -->
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-dependency-plugin</artifactId>
                <version>2.1</version>
                <executions>
                    <execution>
                        <id>copy</id>
                        <phase>generate-resources</phase>
                        <goals>
                            <goal>copy</goal>
                        </goals>
                        <configuration>
                            <artifactItems>
                                <artifactItem>
                                    <groupId>com.google.jstestdriver</groupId>
                                    <artifactId>jstestdriver</artifactId>
                                    <version>${js-test-driver.version}</version>
                                    <type>jar</type>
                                    <overWrite>true</overWrite>
                                    <destFileName>jsTestDriver.jar</destFileName>
                                </artifactItem>
                            </artifactItems>
                            <outputDirectory>${project.build.directory}/jstestdriver</outputDirectory>
                            <overWriteReleases>false</overWriteReleases>
                            <overWriteSnapshots>true</overWriteSnapshots>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <!-- Copy JavaScript files and Tests files  -->
            <plugin>
                <artifactId>maven-resources-plugin</artifactId>
                <version>2.4.3</version>
                <executions>
                    <execution>
                        <id>copy-main-files</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/test-classes/main-js</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>src/main/webapp/scripts</directory>
                                    <filtering>false</filtering>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                    <execution>
                        <id>copy-test-files</id>
                        <phase>generate-test-resources</phase>
                        <goals>
                            <goal>copy-resources</goal>
                        </goals>
                        <configuration>
                            <outputDirectory>${project.build.directory}/test-classes/test-js</outputDirectory>
                            <resources>
                                <resource>
                                    <directory>src/test/webapp/scripts</directory>
                                    <filtering>false</filtering>
                                </resource>
                            </resources>
                        </configuration>
                    </execution>
                </executions>
            </plugin>

            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-antrun-plugin</artifactId>
                <version>1.6</version>
                <executions>
                    <execution>
                        <phase>test</phase>
                        <goals>
                            <goal>run</goal>
                        </goals>
                        <configuration>
                            <target>
                                <mkdir dir="${project.build.directory}/js-test-results" />
                                <exec executable="${xvfb.location}">
                                    <arg line="java" />
                                    <arg line="-jar ${project.build.directory}/jstestdriver/jsTestDriver.jar" />
                                    <arg line="--port 4220" />
                                    <arg line="--testOutput ${project.build.directory}/js-test-results" />
                                    <arg line="--config ${project.build.directory}/test-classes/jsTestDriver.conf" />
                                    <arg line="--browser ${firefox.location}" />
                                    <arg line="--tests all" />
                                    <arg line="--verbose" />
                                </exec>
                            </target>
                        </configuration>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>
</profile>

Possible problems

Although I cannot predict or fix all problems, I can share the one major problem I ran into with Solaris and the script used to fix that. In Solaris (and could happen to other distros) the xvfb-run script was not available and several of the other libraries did not exist. I first had to download the latest X libraries and place them in their appropriate locations on the CI server. Next, I had to re-engineer the xvfb-run script. Here is a copy of my script (NOTE: This is the solution for my server and this may not work for you)

I created a script that contains:

/usr/openwin/bin/Xvfb :1 screen 0 1280x1024x8 pixdepths 8 24 fbdir /tmp/.X11-vbf &

July 12, 2011

Maven's WAR Overlays: How to manage dependencies

If you're stumbling on this post looking for a way to manage dependencies with WAR overlays, please make sure you check out part one of this blog posting to get the background about how to apply and use WAR overlays in your POM file. This post picks up from the last post and it's assumed that you understand how to apply a WAR to an existing master project.

Quick recap

Why a quick recap? We're going to go into depth on how dependencies and libraries are applied to the master project. In order to have a WAR overlay, you must include the WAR as a dependency to the master project and must be overlayed during the war-plugin package phase.

Where do the dependency libraries go?

As mentioned in part one of this blog posting, files are placed in their parent's locations if it exists. What this means is that if, in your overlay WAR you have a .jsp file in /WEB-INF/welcome.jsp, the master WAR will then have a file named "welcome.jsp" in the /WEB-INF. Furthermore, if there is already a file named "welcome.jsp", that file will not be overwritten. If we extrapolate this a bit more, the libraries in the /WEB-INF/lib folder from the overlay will be placed inside the /WEB-INF/lib folder of the master project. This is great and all except when different versions of libraries exist causing runtime failures. Good news, there are ways to manage the dependencies in both the overlay WAR and the master project.

How to see your main artifact's libraries?

In order to determine what libraries are in which artifact, there are two simple ways. First, determine what libraries are in your master WAR.

mvn clean package 

ls -al target/YOUR_ARTIFACT_WAR/WEB-INF/lib

NOTE: adding "-DskipTests" when added to the above maven command would skip tests and would make this step a bit faster, though, it is strictly optional

Secondly, add the "workDirectory" parameter to the configuration portion of your war-plugin when applying the overlay. Example: <workDirectory>target/extract_here</workDirectory>

In your master project's POM file:
<plugins>
...
...
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
                <workDirectory>target/overlay-war-folder</workDirectory>
                <overlays>
                        <overlay>
                                <groupId>com.yourcompany</groupId>
                                <artifactId>branded-templates</artifactId>
                        </overlay>
                </overlays>
        </configuration>
</plugin>
...
...
</plugins>

This additional property will place each overlay's exploded contents into the specified folder to help debug library conflicts.

How to exclude dependencies/libraries at build time?

It can be a pain to manage the dependencies between each of the overlays, especially if the overlay is not under your development control. One method would be to manage the dependencies within the two POM files by utilizing the "provided" and "optional" dependencies that maven provides. However, if the overlay project can also stand as stand-alone project, this method will not work because the libraries are not considered to be provided or optional when running independently. This is common when trying to test each of the overlays independently. An additional option to control the libraries is to exclude them from the master WAR during the package phase. This is achieved by using the "excludes" parameter in each of the overlay sections. For example, to remove all of the spring libraries, you could exclude "spring*.jar"

<plugins>
...
...
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
                <workDirectory>target/overlay-war-folder</workDirectory>
                <overlays>
                        <overlay>
                                <groupId>com.yourcompany</groupId>
                                <artifactId>branded-templates</artifactId>
                                <excludes>
                                        <exclude>WEB-INF/lib/spring*.jar</exclude>
                                </excludes>
                                <excludes>
                                        <exclude>WEB-INF/lib/log4j*.jar</exclude>
                                </excludes>
                        </overlay>
                </overlays>
        </configuration>
</plugin>
...
...
</plugins>

You might start asking yourself, this is going to get complicated really quickly, and you're right, there are other options.

Skinny vs Fat WAR Overlays

Up until now, all of our overlays have been considered "fat" because they contain all of it's dependencies which get overlaid onto the master WAR file. There is an alternative that is more commonly known as the "skinny WAR". A skinny WAR is a combination between most of this post's tactics. NOTE: The method I am showing you is known as a "skinnier WAR" because the dependency libraries do exist in the overlay's WAR file, however, the classes will be externalized and the libraries can be removed during final packaging of the master project.

Achieving the "skinny WAR"

First, in the overlay project, during the package phase, tell maven to build the project's classes as an extra artifact in addition to the WAR file. This set of classes will become a jar file which will be used as a second dependency in the master project's POM.

In the Overlay's POM file (add to the existing war plugin):
<plugins>
...
...
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
                <attachClasses>true</attachClasses>
        </configuration>
</plugin>
...
...
</plugins>

After successfully installing/deploying the overlay project, we will add the classes/jar file and the WAR dependency to the master project's POM file.

In the master project's POM file:
<dependency>  
   <groupId>com.yourcompany</groupId>  
   <artifactId>branded-templates</artifactId>  
   <type>war</type>  
</dependency>  


<dependencies>
...
...
<dependency>
        <groupId>com.yourcompany</groupId>
        <artifactId>branded-templates</artifactId>
        <version>1.1</version>
        <type>war</type>
</dependency>
<dependency>
        <groupId>com.yourcompany</groupId>
        <artifactId>branded-templates</artifactId>
        <version>1.1</version>
        <type>jar</type>
        <classifier>classes</classifier>
</dependency>
...
...
</dependencies>

And finally, combine the excludes learned in the last example section above to exclude all .jar files from the overlay WAR.
<plugins>
...
...
<plugin>
        <groupId>org.apache.maven.plugins</groupId>
        <artifactId>maven-war-plugin</artifactId>
        <version>2.1.1</version>
        <configuration>
                <workDirectory>target/overlay-war-folder</workDirectory>
                <overlays>
                        <overlay>
                                <groupId>com.yourcompany</groupId>
                                <artifactId>branded-templates</artifactId>
                                <excludes>
                                        <exclude>WEB-INF/lib/*.jar</exclude>
                                </excludes>
                        </overlay>
                </overlays>
        </configuration>
</plugin>
...
...
</plugins>

NOTE: Dependencies that exist in the Overlay will need to be added as dependencies to the master project because they have been removed from the overlay's /WEB-INF/lib folder. This is the main known risk of using skinny war overlays

Congratulations!

You now have a project with limited library conflicts and can better manager the main project and it's stability.

Thank you for reading part one and this post, part two. Please feel free too leave comments or questions and I will do my best to answer them in a timely manor.

June 27, 2011

Maven's WAR Overlay: What are WAR Overlays?

What is a WAR overlay?

def: Overlays are used to share common resources across multiple web applications. The dependencies of a WAR project are collected in WEB-INF/lib, except for WAR artifacts which are overlayed on to the WAR project itself. http://maven.apache.org/plugins/maven-war-plugin/overlays.html

Why and where would you use a WAR overlay?

WAR overlays make up one more tool in your arsenal to combat repetitive code and/or resources. Overlays allow you to combine many similar resources, for example, images, CSS and JavaScript files all into one project. At build time, the overlay will be predictably merged into a master WAR project. A great example of a WAR overlay is a branded website, combined with Sitemesh decorators. Imagine you have a branded company website which consists of multiple web applications. An industry standard approach would be to utilize Sitemesh decorators to decorate the application with brand specific styles and allow each of your applications to focus on the specific purpose of that application. At this time, this blog post will not go into depth on Sitemesh decorators, however, more details are available at http://www.sitemesh.org/.

Leading on with our example, each of the decorators will need a set of CSS, images and possibly JavaScript files to define the application's branded style. If your applications are to be deployed to production to look and act as one cohesive branded site, you will need to make sure that the images, CSS, JavaScript, decorator files and JSP files are all the same in each of your applications in order to keep a consistant brand across your applications. Well, this is duplicated code (currently ranked in my top 5 biggest "code avoid-at-all-cost" list).

How do you solve code duplication using WAR Overlays?

One approach to reducing the amount of duplicated code is to create a separate maven project with the packaging target type of WAR. Then in each of your application's pom.xml file, include the overlay project as a dependency and construct it as an overlay using the maven-war-plugin. This is known as a Fat WAR Overlay. (NOTE: In the next post I will cover other types of WAR overlays to help mitigate transient dependency problems)

More specifics please!

First, let's create a new maven WAR project and name it "branded-templates". (NOTE: This post is not intended to be a comprehensive guide on creating a maven project, only adding to an already working maven project. If you need more assistance, please refer to some of my other post or maven's website for more guidance.)

In our new branded-templates application, we will need to fill this with our resources. Here is my recommended locations to place web-related resources within your project:
  • /src/main/webapp/styles – used to hold your CSS files, (example: brand.css, reset.css, etc)
  • /src/main/webapp/scripts/ – used to hold brand-wide JavaScript files, (example: jquery.1.x.x-min.js, jquery-my-company-validation.js, etc
  • /src/main/webapp/images/ – used to hold brand-wide images, (example: company-logo.png, facebook-icon.png, company-background.jpg, etc)

Naming for now will not be important, but you will have to make decisions here because you have two naming options here that will have development implications. You can name your resources and/or folders the same in both of your applications or you can name them different to get different desired results. How is this going to impact my design and why do you need to think about this? (Answer coming after we add to the containing webapp project, keep following!)

Let's continue. We can now install your branded-templates WAR overlay to your local maven repository:

mvn clean install

If all is good, you should see a "Successful build" along with a 1.0-SNAPSHOT for your WAR's version.

Next, let's crate a new main application called "main-webapp". If you have an existing webapp, feel free to use that and work along with this example. Inside of the main-webapp's pom.xml file, we need to add a plugin to build the two WARs into one WAR and add the overlay as a dependency to the project.

Add to your dependencies section:
<dependency>
   <groupId>com.yourcompany</groupId>
   <artifactId>branded-templates</artifactId>
   <type>war</type>
</dependency>

Note that we are adding this as a WAR type, do not leave the type blank.

Add to your plugins section:
<plugin>
 <groupId>org.apache.maven.plugins</groupId>
 <artifactId>maven-war-plugin</artifactId>
 <version>2.1.1</version>
 <configuration>
   <overlays>
     <overlay>
       <groupId>com.yourcompany</groupId>
       <artifactId>branded-templates</artifactId>
     </overlay>
   </overlays>
 </configuration>
</plugin>

Next, we've added the war-plugin which will take our application and build a WAR. If you're familiar with maven's artifact types, you'll already know that your project builds a WAR already, so why the plugin? The plugin is where the "magic" of the overlay happens. This is where the overlay is placed into the main-webapp's WAR file.

How many WAR files will I have?

When using a WAR overlay, you will start with two or more WAR files (presumably one or more overlays and a main-webapp), but will ultimately end up with one WAR artifact (your main-webapp) in which to deploy to your environment. Though it is completely possible to run your overlay as a standalone application, I recommended you read my next post (coming soon) on dependency management within overlay projects to avoid dependency problems in your main-webapp project.

So what happens when there is a conflicting resource?

As you can imagine, there can be conflicts between overlay's resource names and the main project's resources. This is what I asked you to think about from the question posed above. If you have the same location and named file in your overlay application as you do in your main application, the overlay version will be ignored for the main application's version. This can work for or against you, so think about how you might use both. How would you want to leverage the customization abilities in the main webapp.

Note, if you plan on utilizing Sitemesh decorators, it's not a bad idea to put your decorators.xml and sitemesh.xml file in your WEB-INF folder and the corresponding JSPs in the WEB-INF folder (suggest placing them in /WEB-INF/decorators folder). Remember to add the sitemesh web.xml information into your containing main-webapp's web.xml!

One more gotcha that might arise out of this would be dealing with the web.xml. The overlay's web.xml file is NOT copied over nor is it merged in any way, so if your overlay has specific web.xml file information you can either manually add it to the web.xml or use a <include file="/WEB-INF/web-overlay.xml"> tag where you've added the contents to a file named web-overlay.xml to the overlay war project.

Tips for rapid development

At this point you should have an overlay and a working master webapp leveraging that WAR overlay. Since you have your resources in the overlay and you are probably working in your main-webapp, and it is probably becoming a pain to keep deploying a new version of your overlay project, you can simply refer to the SNAPSHOT version of your overlay inside your main-webapp pom.xml file and do a local install of your overlay.

Leaving notes

Follow my blog and get the next blog post dealing with dependency management with overlays.

Feel free to leave comments or questions and I'll do my best to get back to you in a reasonable time frame. I would like to thank Dynacron Group (my consulting firm) for allowing me some time to work on this blog post!

May 31, 2011

Maven + JavaScript Unit Test using JsTestDriver

What are we doing?
You wake up to the annoying Justin Timberlake ringtone you set for the office only to discover it's the site is on fire" call. A quick glance at the alarm clock and you realize it's 3am. It's this time when you decide that you need to get some test coverage on your Java Script.

What is Unit Testing?
Unit Testing is a method by which individual units of source code are tested to determine if they are fit for use. A unit is the smallest testable part of an application and in JavaScript, typically it is a function.

Why would you want to unit test JavaScript?
JavaScript has grown from the little scripts that would change button images on rollover to a fully functional, and often application critical, complex applications. There are many interactions with DOM objects, AJAX request/response and user events that make testing your code a must-have. Unit testing will give you the the confidence to add new features and change existing code. In addition, unit test self-document the code so you can be sure that the rad new function you wrote does exactly what you want it to.

What is covered
What the rest of this blog will describe is one approach to integrate maven and JsTestDriver for local development. In a near future blog, this approach will be extended to use on a Linux/Unix based system for a continuous integration system.

Approach
There are many approaches to unit testing JavaScript (QUnit being one of the many), however, most of the unit test frameworks do not easily allow you to integrate your unit test as a part of a maven testing cycle. The one problem with all JavaScript unit testing frameworks is that you need to have a place to run your test. At the time of this blog, there is not a "headless" browser, thus you (or your framework) must open a browser, accept JavaScript loading and running the test cases while providing repeatable testing output. This blog series will discuss a few different methods to run unit test, including a method to run in a virtual framebuffer for continuous integration systems that do not allow you to open a browser in the default display.

What is JsTestDriver?
JsTestDriver is an small application that controls your test cases, your source files and the interaction between the browsers you choose to test against. JsTestDriver allows you to only load the source files you want, along with only the DOM elements you need to unit test your source code, in other words, the browser ONLY loads your JavaScript files and optionally HTML and/or DOM elements needed by your test.

How to integrate with Maven
The approach here is to add JavaScript testing via a profile, so you can create a CI build specifically for your JavaScript unit test.

  1. JsTestDriver Setup

    This blog will not discuss in depth how to setup and run JsTestDriver. For more in-depth information, please visit http://code.google.com/p/js-test-driver/ website. This blog is using JsTestDriver 1.3.2.

    In order to load your test scripts, you must create a configuration file. For a maven application, place your configuration file in your "/resources" folder.

    Here is an example properties file (jsTestDriver.conf)

    server: http://localhost:4220
    
    load:
     # Header JS Files, all files are loaded top-down
     - main-js/lib/jquery.min.js
     # Body JS files
     - main-js/Person.js
     # Test files
     - test-js/*.js
    
    exclude:
     - main-js/exclude.js
    
    

    The above configuration file assumes your file structure is similar to:
    /src/main/webapp/scripts/*.js
    /src/test/webapp/resources/jsTestDriver.conf
    /src/test/webapp/scripts/*.js

    (NOTE: You might notice that the file path is different than the "main-js". Keep reading!)

  2. Maven Dependencies and Profile

    Next, let's add the dependencies to the POM file in your project.

    Add the following to your "dependencies" section in your pom.xml
    <dependency>
     <groupid>com.google.jstestdriver</groupId>
     <artifactid>jstestdriver</artifactId>
     <version>1.2.2</version>
     <scope>test</scope>
     <type>jar</type>
    </dependency>
    
    

    Next, we need to add a profile, which will do quite a few things. First, the JsTestDriver jar file will be copied from the repository to a known folder in the /target folder for use when running the JsTestDriver server. Next, we need to copy the JavaScript files from your project's location to a known folder in the "/test" folder, and also to match the "main-js/*" and "test-js/*" folder structures that we setup in the jsTestDriver.conf file. Lastly, we need to startup the JsTestDriver server so we can run our test.

    Here is a profile to start with:

    <profile>
        <id>local-jstests</id>
        <build>
            <plugins>
    
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-dependency-plugin</artifactId>
                    <version>2.1</version>
                    <executions>
                        <execution>
                            <id>copy</id>
                            <phase>generate-resources</phase>
                            <goals>
                                <goal>copy</goal>
                            </goals>
                            <configuration>
                                <artifactItems>
                                    <artifactItem>
                                        <groupId>com.google.jstestdriver</groupId>
                                        <artifactId>jstestdriver</artifactId>
                                        <version>${js-test-driver.version}</version>
                                        <type>jar</type>
                                        <overWrite>true</overWrite>
                                        <destFileName>jsTestDriver.jar</destFileName>
                                    </artifactItem>
                                </artifactItems>
                                <outputDirectory>${project.build.directory}/jstestdriver</outputDirectory>
                                <overWriteReleases>false</overWriteReleases>
                                <overWriteSnapshots>true</overWriteSnapshots>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
    
                <plugin>
                    <groupId>org.apache.maven.plugins</groupId>
                    <artifactId>maven-surefire-plugin</artifactId>
                    <configuration>
                        <skipTests>true</skipTests>
                    </configuration>
                </plugin>
    
                <plugin>
                    <artifactId>maven-resources-plugin</artifactId>
                    <version>2.4.3</version>
                    <executions>
                        <execution>
                            <id>copy-main-files</id>
                            <phase>generate-test-resources</phase>
                            <goals>
                                <goal>copy-resources</goal>
                            </goals>
                            <configuration>
                                <outputDirectory>${basedir}/target/test-classes/main-js</outputDirectory>
                                <resources>
                                    <resource>
                                        <directory>src/main/webapp/scripts</directory>
                                        <filtering>false</filtering>
                                    </resource>
                                </resources>
                            </configuration>
                        </execution>
                        <execution>
                            <id>copy-test-files</id>
                            <phase>generate-test-resources</phase>
                            <goals>
                                <goal>copy-resources</goal>
                            </goals>
                            <configuration>
                                <outputDirectory>${basedir}/target/test-classes/test-js</outputDirectory>
                                <resources>
                                    <resource>
                                        <directory>src/test/webapp/scripts</directory>
                                        <filtering>false</filtering>
                                    </resource>
                                </resources>
                            </configuration>
                        </execution>
                    </executions>
                </plugin>
    
                <plugin>
                    <artifactId>maven-antrun-plugin</artifactId>
                    <version>1.6</version>
                    <executions>
                        <execution>
                            <phase>test</phase>
                            <configuration>
    
                                <target>
                                    <!-- make output folder -->
                                    <mkdir dir="${basedir}/target/js-test-results"/>
                                    <!-- start jsTestDriver server -->
                                    <exec executable="java">
                                        <arg line="-jar ${basedir}/target/jstestdriver/jsTestDriver.jar"/>
                                        <arg line="--port 4220"/>
                                        <!-- Possibly set this up to point to the same location of your surefire output -->
                                        <arg line="--testOutput ${basedir}/target/js-test-results"/>
                                        <arg line="--config ${basedir}/target/test-classes/jsTestDriver.conf"/>
                                        <arg line="--browser /opt/google/chrome/google-chrome"/>
                                        <arg line="--tests all"/>
                                        <arg line="--verbose"/>
                                        <arg line="--captureConsole"/>
                                        <arg line="--preloadFiles"/>
                                    </exec>
                                </target>
                            </configuration>
                            <goals>
                                <goal>run</goal>
                            </goals>
                        </execution>
                    </executions>
    
                </plugin>
            </plugins>
        </build>
    </profile>
    
    

  3. Sample Unit Test

    Now you need to have a sample JavaScript file and a corresponding unit test.

    Here is a simple JavaScript file (/src/main/webapp/scripts/Person.js)

    NAMESPACE = {};
     
    NAMESPACE.Person = function(initObj) {
     this.name = initObj.name;
     this.age = initObj.age;
     this.address = initObj.address;
     this.sex = initObj.sex;
     this.significantOther = initObj.significantOther;
     
     function setSignificantOther(person) {
      this.significantOther = person;
     }
     
     this.setSignificantOther = setSignificantOther;
    }
    

    And a sample test case might be: (/src/test/webapp/scripts/PersonTest.js

    TestCase("PersonTest", {
     testA:function(){
      expectAsserts(1);
    
       var peterGriffin = new NAMESPACE.Person({
        name: "Peter Griffin",
        age: 43,
        address: {
         address1: "31 Spooner Street",
         address2: "",
         city: "Quhog",
         state:"RI"
        },
        sex: "Male",
        significantOther: null
       });
       
       assertNotNull("The variable should not be null", peterGriffin);
      };
     }
    });
    
    
  4. Running your test

    In order to run your test, you can now type:

    mvn clean test -Plocal-js


Please stay tuned for the expansion to this on how to run in a virtual framebuffer for a CI environment (and of course, updates for found errors)