Tuesday, November 17, 2009

Simple Web Tool Building With Maven (The Woot-Off Tracker Revisited)

One of my favorite tools of the past year was the Woot-Off tracker I developed just over a year ago. In my mind, one of the problems this tool had was that I needed to start up a proxy server to host make calls on behalf of the AJAX Javascript. I needed to find some way to make this easier. Additionally, I had run into an issue when I actually tried to run the tool behind a firewall proxy.



Since discovering the power of Maven, I thought I'd look into seeing what was available for the Winstone servlet engine I used on the first iteration of this tool. Conviently, there was the winstone-maven-plugin, a mojo which, among other things, built an executable jar for winstone with your war already deployed.



This was perfect for my needs. So (along with converting to apache's http-client) I set out on a new maven adventure. My pom looked like this:





<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/maven-v4_0_0.xsd">
<modelVersion>4.0.0</modelVersion>
<groupId>com.twofourone</groupId>
<artifactId>woot.tracker.proxy</artifactId>
<packaging>war</packaging>
<version>1.0-SNAPSHOT</version>
<name>woot.tracker.proxy</name>
<url>http://twofourone.blogspot.coom</url>
<dependencies>

<dependency>
<groupId>javax.servlet</groupId>
<artifactId>servlet-api</artifactId>
<version>2.4</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>javax.servlet.jsp</groupId>
<artifactId>jsp-api</artifactId>
<version>2.0</version>
<scope>provided</scope>
</dependency>

<dependency>
<groupId>junit</groupId>
<artifactId>junit</artifactId>
<version>4.6</version>
<scope>test</scope>
</dependency>

<dependency>
<groupId>org.apache.httpcomponents</groupId>
<artifactId>httpclient</artifactId>
<version>4.0</version>
</dependency>

</dependencies>
<build>
<plugins>
<plugin>
<groupId>org.apache.maven.plugins</groupId>
<artifactId>maven-compiler-plugin</artifactId>
<version>2.0.2</version>
<configuration>
<source>1.5</source>
<target>1.5</target>
</configuration>
</plugin>
<plugin>
<groupId>net.sf.alchim</groupId>
<artifactId>winstone-maven-plugin</artifactId>
<executions>
<execution>
<goals>
<goal>embed</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
<plugin>
<groupId>net.sf.alchim</groupId>
<artifactId>winstone-maven-plugin</artifactId>
<version>1.2</version>
<configuration>
<cmdLineOptions>
<property>
<name>httpPort</name>
<value>8480</value>
</property>
<property>
<name>ajp13Port</name>
<value>-1</value>
</property>
<property>
<name>controlPort</name>
<value>-1</value>
</property>
<property>
<name>directoryListings</name>
<value>false</value>
</property>
<property>
<name>useInvoker</name>
<value>false</value>
</property>
</cmdLineOptions>
</configuration>
<executions>
<execution>
<goals>
<goal>embed</goal>
</goals>
<phase>package</phase>
</execution>
</executions>
</plugin>
</plugins>
<finalName>woot.tracker.proxy</finalName>
</build>
</project>


We have two setups for winstone. The first goal, embed, tells Maven to stick winstone into the jar during the package phase. By default, this step will create a jar with the same name as your project and throw -standalone on the end. The second setup tells winstone what port it should run on, etc. These could potentially be in a single setup, just haven't tried it yet.



In order to run the proxy, you simply execute the jar from a command prompt and you'll have a war deployed to port 8480.



How has our proxy changed? In order to support a firewall call, I decided to use Apache's HttpClient. Adding the dependency for HttpClient to the pom automagically downloads all dependencies of HttpClient for use by the war. Our new code looks like this:




package com.twofourone.woot.tracker.proxy;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.OutputStream;
import javax.servlet.ServletException;
import javax.servlet.http.HttpServlet;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;

import org.apache.http.HttpEntity;
import org.apache.http.HttpHost;
import org.apache.http.HttpResponse;
import org.apache.http.client.HttpClient;
import org.apache.http.client.methods.HttpGet;
import org.apache.http.conn.params.ConnRoutePNames;
import org.apache.http.impl.client.DefaultHttpClient;

/**
* Proxy Servlet to redirect requests to the Woot site.
* @author mcornell
*/
public class WootProxy extends HttpServlet {

static final long serialVersionUID = 1L;

private HttpClient httpClient = new DefaultHttpClient();
private HttpGet wootGet;

/**
* Sets up the HttpClient with proxy info if provided. Sets up the URL for Woot.
*/
@Override
public void init() throws ServletException {
String proxyHost = System.getProperty("proxyHost", "");

String proxyPort = System.getProperty("proxyPort", "");

if (proxyHost.length() > 0 && proxyPort.length() > 0) {
HttpHost proxy = new HttpHost(proxyHost, Integer.parseInt(proxyPort));
httpClient.getParams().setParameter(ConnRoutePNames.DEFAULT_PROXY, proxy);
}

String contentObj = "http://www.woot.com/salerss.aspx";
wootGet = new HttpGet(contentObj);
}


/**
* Processes requests for both HTTP GET and POST methods.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
protected void processRequest(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {

HttpResponse wootResponse = httpClient.execute(wootGet);

HttpEntity wootEntity = wootResponse.getEntity();

response.setContentType(wootEntity.getContentType().toString());
// Get and read the input stream.
StringBuffer buffer = new StringBuffer();

BufferedReader din =
new BufferedReader(new InputStreamReader(wootEntity.getContent()));

String s;
while ((s = din.readLine()) != null) {
buffer.append(s);
}
din.close();

// Now write the bytes out to the client.
byte[] contentBytes = buffer.toString().getBytes();
OutputStream out = response.getOutputStream();
out.write(contentBytes, 0, contentBytes.length);
out.flush();
out.close();
}

/**
* Handles the HTTP GET method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doGet(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Handles the HTTP POST method.
* @param request servlet request
* @param response servlet response
* @throws ServletException if a servlet-specific error occurs
* @throws IOException if an I/O error occurs
*/
@Override
protected void doPost(HttpServletRequest request, HttpServletResponse response)
throws ServletException, IOException {
processRequest(request, response);
}

/**
* Returns a short description of the servlet.
* @return a String containing servlet description
*/
@Override
public String getServletInfo() {
return "Woot Tracker Proxy";
}
}


The code has changed to setup the client once and initialize it once. Additionally, there is the code provided by the HttpClient project that adds a proxy to the client. One additional change needs to be the ability to track a Kids.Woot-Off URL in case it's a different kind of woot-off.



The HTML/Javascript remains the same.

Friday, October 30, 2009

Converting to the Ultimate Java Build Stack

Over the past 6 months, I've had the pleasure and pain of working with Maven. Even with all of the pain, I've found myself a fan of the technology, assuming, of course, that you have the appropriate Maven stack set up.

During my time at the client, I setup Hudson, Nexus, and Sonar as outlined in Christopher Judd's blog post. We had actually already decided upon Hudson and Nexus through discovery before Judd published his post. Sonar was a nice discovery we added to the mix a bit later. Having setup this stack, I thought I'd offer some thoughts on my experiences.

I cannot stress how important having Nexus (or Artifactory) is to keeping your Maven projects running smoothly and effectively. Having a repository management tool and proxy allows for all of your developers (and Hudsons) to simply point at the proxy and download everything required for a build quickly. It also provides a place for you to host your third party jars from apps that have been purchased by your organization. Not everything is open source of course.

The steps are simple enough, install Nexus and setup your settings.xml file to point to it. For developers, they can use their own settings.xml file. For Hudson, you may want to modify the settings.xml in your Maven install (especially if you're running on Windows). You might need to add a mirror for Java.net's Maven 2 repository. You might not. It depends upon how up to date Maven's central repository is.

Sonar was a nice surprise. Providing PMD, Checkstyle and Cobertura results in a nice visual manner that allowed for quick feedback was surprisingly effective. As a developer, I found it to be my favorite part of the switch to Maven. Sure, Maven provides this information as part of its site goal, however, it doesn't present it like this.

Obviously there are huge gains to be made by integrating Hudson and getting it to build upon version control commits, but I found more intriguing was its ability to trigger downstream builds. When I built the commons library for the app, every maven artifact that used the commons library was triggered for build. Numerous times, I discovered that I or someone else had broken something in a downstream library due to an API fix. That instant feedback (through RSS feeds) was an invaluable time saver.

What about Maven itself? What does it gain you? In the case I was working on, tremendous build granularity. The original build used Ant, and was built in a monolithic manner. Each component was built by the single script and bundled together. One could argue it wasn't the best ant script in the world, and they'd be right. Maven allowed us to break these dependencies apart. So now, instead of having one monolithic code base which included what were really five separate artifacts, we now had five separate projects, each self contained using dependencies to draw in the needed artifacts. As such the WAR and EAR builds ended up being very simple where they just drew in their needed dependencies and just worked.

Additionally, Maven offered unit testing at a very cheap cost. The initial code base had no unit testing. The developers were keen to use the debugger to test scenarios, which is needed to understand the mass amounts of reflection which were in use. However, that's only helpful in understanding the code. There should have been unit tests which were stood up to test each component on its own. Maven automagically gives you unit testing with the surefire plugin enabled by default. Make sure JUnit is a test dependency and start writing tests into the test source tree. Keep your tests in the same package as your code and start gaining coverage. In between adding new features to the code base, I was able to increase code coverage on the base level components from 0 to 40%. Obviously there's still a long way to go, but this was a vast improvement!

One thing I will note about Maven is the tooling. There is tremendous tooling available in the m2eclipse plugin for Eclipse. Netbeans 6.5 and above opens Maven projects without needing to do much of anything. It just opens the directory containing the pom and imports the project. If you're moving to Maven, make sure that your development tools can support it. As an example, RAD 7.0.x is Eclipse 3.2. As such the m2eclipse plugin does not work for it. If you must use RAD 7.0.x, I would advise building your project outside of RAD. Use a more up to date IDE to get the projects started and structures built. Then use the maven-eclipse-plugin to build the proper project eclipse artifacts for your projects. It will still be a challenge to build proper EAR and WAR projects for RAD. Maven has a lot of help for this sort of pain. However it can only be applied if you use the proper tools.

I've got a bit more to say about Maven, especially with my experiences of converting projects to Maven, the various pain and joy points I experienced and overall satisfaction. But I'll leave that for another post. Until then, I strongly suggest that if you have new development and you are looking for a pure Java based project build management tool, that you should take a long hard look at Maven.

Monday, September 14, 2009

Wierdness with Maven's site-deploy and Hudson

So, for the past five months or so, I've been working on porting some Java projects to Maven and the Ultimate Enterprise Java Build System.


A quick aside, Sonar is a very underrated tool. You should try it. I find the reporting very easy to use and very handy for working through potential coding mistakes in the legacy code I've been working with.


One of those Java projects is a large multi-module Maven build. It has five different modules, three Jar modules, a War module and *gasp* an Ear module. It's been a bit of a beast to get going, but it seems to be working just fine.


On Monday, I began to perform releases to move from Maven SNAPSHOTs to actual real version numbers. Ah, the joys of SCM release management. Thankfully, all of this pain ends up being worth it to easily tag and lock the code down using the maven-release-plugin. There were a couple of bumps, but it wasn't bad.

The fun began when I built my Hudson jobs to perform release builds. These jobs simply pulled the tagged release and performed the following goals:


clean install site-deploy


Obviously, I used Hudson's deploy step for the artifacts rather than Maven's. And everything was fine, until I got this:


[INFO] ------------------------------------------------------------------------
[ERROR] BUILD ERROR
[INFO] ------------------------------------------------------------------------
[INFO] Failed to resolve artifact.

Oops, what's that about? A little more background. This particular maven build was a multimodule build to create jar, war and ear files. The artifact it couldn't resolve was the first jar it built! Scanning the log file, it became clear that the job was not installing the built jar to the repository.


I don't know why that is, and am not certain if it's even worth opening a trouble ticket since the fix ended up being very simple. It seems that site-deploy was causing issues with the flow of the job within Hudson. By removing site-deploy the build ended up being successful. The jar was compiled, tested and installed before the next module's build was invoked.


I still wanted to build the site, so I investigated the M2 Extra Steps Plugin. This tool allows you to add multiple maven steps before and after the main job. I added the site-deploy goal to run after the main install goal and the sites were generated!

ShareThis