Building a modern Java app with Eclipse, Maven, OSGi, and Spring DM (Part 1)

July 8th, 2009  |  Published in development  |  5 Comments

Last week Michael Nygard tweeted about difficulties with Eclipse, Maven, OSGi, and Spring DM. Given that Michael and others have expressed interest in hearing how we’ve been using all of those technologies when developing VMware’s forthcoming vCloud product, I thought I’d try to go through it over the next week or two in a series of blog posts. Today’s post will provide some of the details about our base Maven setup.

Note: I won’t be talking about the specifics of our product, so if you’d like more details please consult the presentation from my colleague Orran Krieger. I’m also not going to touch much on our deployment work; for that see my other colleague Stephen Evanchik’s blog or the Eclipse Integration for Karaf project he started on FUSE Forge.

Project layout and building the product

The product codebase is almost entirely Java and when we first started writing code last year it seemed to make sense to use a tool that understood Java and was able to help us resolve third-party dependencies. At the time we weren’t really aware of Ant+Ivy, so we opted to go with Maven. It was also nice that Maven follows the convention-over-configuration approach which made it very easy to create new modules and quickly get new developers up to speed. One downside at the time was that the Sonatype folks were still working on their Maven book, so we ended up having to figure out a number of Maven best practices on our own, which resulted in some frustration early on until we got over the learning curve. Today we have about 120 modules that are part of the build, and 1-2 dozen additional modules that are not part of the regular process.

We have a single master POM that defines the project defaults (including artifact versions and plug-in configurations). The rest of the modules are organized into subsystems, and each subsystem has its own POM to allow us to build them in isolation if we wish (in some cases there are inter-subsystem dependencies that prevent us from doing so). For the final deliverable we rely on Maven assemblies to collect the appropriate artifacts and package them into a tarball suitable for distribution.

One other important distinction is that we always download artifacts into a local repository that is checked into Perforce (the SCM system we use) and run Maven in offline mode. This allows us to reproduce any build based on the Perforce changelist number and also means the team doesn’t spend all day downloading artifacts just to do a build.

Recommendations

  1. Place parent POMs in a sibling directory (i.e. ../foo-parent) instead of in the parent directory (../) as Eclipse/m2eclipse seems to handle the nested projects slightly better. We originally ran into problems where Eclipse would start shifting files and output artifacts around when the parent POM wasn’t in its own directory.
  2. Define variables in the master POM to capture artifact versions. This will allow you to update the value in one place and have the change automatically propagate throughout the system. There is nothing more frustrating than having to search and replace version strings through 120 modules and in different scopes. Multiply that by the number of artifacts that comprise Spring or Spring DM and you’ll soon be begging for a drink.
  3. Take advantage of the dependencyManagement and pluginManagement elements so artifact versions don’t need to be specified in child POMs.
  4. Utilize profiles to pull out processes that don’t need to be executed all of the time. We originally generated some JAXB and WSDL stubs every time until we eventually moved those goals into deactivated profiles for the few times they actually needed to be changed. We also started to do this for the MANIFEST.MF files, which I’ll touch on more in a future post.
  5. Don’t use snapshot versions of artifacts unless absolutely necessary. They make it extremely difficult (if not impossible) to produce repeatable builds. We got in the habit of disabling snapshots in our repository definitions to make sure they didn’t slip in.
  6. Don’t use version ranges for dependencies if possible; you want to be able to recreate a build exactly without having to guess what version of an artifact was pulled from the repository at the time of the original build. If you’re using an offline repository could be slightly easier because you have a static snapshot of the repository and can sync to a particular changelist number (referring to Perforce in particular).

Tags: , , , ,

Responses

  1. Xqnstyl says:

    July 10th, 2009 at 12:52 pm (#)

    Ah, interesting approach to version your Maven repository. Do you use a repository manager at all, like Artifactory, Archiva or Nexus?

    When I used Maven with Perforce in the past, we had to take great care when merging pom.xml files between branches. How do you handle merging pom.xml files between branches where you want the version to stay different, especially after performing a Maven release?

    I see Perforce has a KB article on Maven integration: http://kb.perforce.com/?article=1023. What’s your take on it?

    Cheers!

  2. Victor says:

    July 24th, 2009 at 3:49 am (#)

    Can you show your working examples of Recommendations #2 and #3, please?

  3. kyle says:

    July 24th, 2009 at 12:08 pm (#)

    @Victor: I’ll try and post something this evening.

    @Xqnstyl: We don’t use a repository manager. The original (local) repository came from one developer’s machine and as new artifacts are needed, we have a couple of simple scripts that will fetch the artifact, its POM, the source if available and then add it to Perforce.

    One key note is that we do not use Maven for performing a release. VMware has its own build and release process (which just happens to call `mvn package` at a particular step). We also haven’t adopted the process of revving individual modules in the project and having people depend on them. The versions are currently all in lockstep and you always depend on the current version (sadly).

    So typically when work is done (even in a branch), it’s the code and the contents of the POM that change, not the version associated with it.

    As for the Maven SCM support for Perforce, we don’t use it. All devs have the p4 client as well as P4WSAD on their systems to edit and submit changesets. Because we don’t use Maven to do releases, we haven’t had a need to try the plugin. From my cursory reading of the page you linked to, I don’t see another case that would be relevant to us.

    ks

  4. Maven and OSGi Roundup « Knowledge Networks says:

    June 17th, 2010 at 7:16 pm (#)

    [...] Presentation giving a good overview (but 2 years old) Best practices from a project experience (Part 1, Part 2, Part 3, Part [...]

  5. James Percent says:

    March 5th, 2011 at 9:55 pm (#)

    I wrote a tool called auto-builder. It introspects PDE-based projects and generates Ant build files; it supports transitive closure over dependencies and all that jazz.

    I posted a write-up here. I wrote it because the Maven tools I played with, when integrated with PDE, didn’t “just work.” Basically, I wanted to do coding in PDE and have a Hudson-based CI without any fuss in between.

    Generating Ant files is nice because it gives you all the benefits of a declarative build tool, but it leaves you with a procedural description of what it is doing.

    I am looking for more PDE-based projects to test it on. There are a couple RFC-0112 Bundle repositories around, and I have some code for downloading dependencies. If anyone is interested, then I could integrate dependencies download with auto-builder.

    James Percent
    james@empty-set.net