Friday, May 22, 2009

Spring and OSGi Integration Testing with Maven

Spring has a great test framework for integration testing OSGi bundles in its Spring Dynamic Modules for OSGi&tm; Service Platforms code (Spring-DM for short). This cool piece of code will load up an OSGi container with the Spring framework, and the Spring-DM code and run your JUnit integration test inside the container for you. This is a very handy and powerful tool to have.

What you must do to make this work is extend an abstract base class (like org.springframework.osgi.AbstractConfigurableBundleCreatorTests) and override one of two methods to specify the additional bundles to install into the OSGi container. If you are using Maven there are great features in that class hierarchy for resolving the required bundles in your local maven repository. This is extremely convenient!

As great as this is, there are a couple of things I am not completely happy with there:

  1. You need to specify the bundles being tested, and all its dependencies including versions in code. That information is already in the POM file for the project, which means I now have to maintain the information in two places.
  2. The Spring version and Spring-DM version are specified by the test code. Sometimes in development I want to change and mix-and match these versions, and can't do that unless I override some other methods that also specify the base bundles to load into the framework.

A good solution to these problems would be to use the Maven POM as the source of all the bundles and their versions to load. Then there is only one place you need to maintain that information, and when you change the versions of something in the POM, you do not need to edit the test classes to match. Using the Maven POM we could get all of the transient dependencies of our test project and have the test framework load everything required without any additional coding.

Use Maven

Fortunately the Spring code is very flexible and configurable (and nicely documented). So we can subclass the AbstractConfigurableBundleCreatorTests class and add the features we want to it. First, though, we need to have a list of all the transient dependencies of our project. It is probably possible to use the maven libraries to process the POM and figure this out for us, but an even easier solution is to just have the Maven build generate what we need into a text file, and read that in our new class. The Maven dependency plugin suits this purpose perfectly. Just add the plugin to your test project POM as follows:

    <build>
        <plugins>
             <plugin>
                  <groupId>org.apache.maven.plugins</groupId>
                  <artifactId>maven-dependency-plugin</artifactId>
                  <executions>
                       <execution>
                           <id>list-dependencies</id>
                           <phase>generate-sources</phase>
                           <goals>
                                <goal>resolve</goal>
                           </goals>
                           <configuration>
                                <outputFile>target/dependencies.list</outputFile>
                           </configuration>
                       </execution>
                  </executions>
             </plugin>
        </plugins>
    </build>

This generates all the transitive dependencies of any scope into a file called target/dependencies.list in the project. Now we can use that file to tell the Spring code what bundles to load into the integration test OSGi container.

Subclass

The Spring AbstractConfigurableBundleCreatorTests class loads the base bundles into the OSGi environment. It gets the list of base bundles from some resource files in the jar itself. We want to override that behaviour to have Spring only load the bundles specified in the dependency list output above. We need to subclass AbstractConfigurableBundleCreatorTests and override two methods. First override the getTestingFrameworkBundlesConfiguration() method to return a Spring resource pointing to our dependency list.

/**
 * Get a resource containing the list of dependencies for the current bundle.
 * The dependencies file could be hand crafted, but it is better to use the
 * Maven dependency plugin to generate this file from the POM contents. Override
 * this method to change the file name to something other than the default.
 * @return the dependencies list file as generated by the Maven dependency plugin. 
 * @see org.springframework.osgi.test.AbstractDependencyManagerTests#getTestingFrameworkBundlesConfiguration()
 */
@Override
protected Resource getTestingFrameworkBundlesConfiguration() {
 return new FileSystemResource(DEFAULT_DPENDENCIES_LIST_FILENAME);
}
Then override the getTestFrameworkBundlesNames() method to parse that file and return the bundles to be loaded. Spring expects to see an array of Strings returned from this method where each string is a bundle to load, in the form "groupId,artifactId,version". The output of the dependency plugin is close to this, but we do need to convert it.
/**
 * Returns the names of the test bundles to load. Overrides the spring method
 * to replace the properties file configuration with configuration from the
 * output of the Maven dependency plugin.
 * @return the bundle names as CSV values containing the maven groupId, artifactId, and version.
 * @see org.springframework.osgi.test.AbstractDependencyManagerTests#getTestFrameworkBundlesNames()
 * @see AbstractInforBundleTests#getTestingFrameworkBundlesConfiguration();
 * @throws IllegalArgumentException if the dependency list resource retrieved from the
 *   {@link #getTestingFrameworkBundlesConfiguration()} method throws an IOException.
 */
@Override
protected String[] getTestFrameworkBundlesNames() {
    List<MavenArtifact> artifacts = null;
    try {
        artifacts = MavenDependencyListParser.parseDependencies(getTestingFrameworkBundlesConfiguration());
    } catch (IOException e) {
        String error = "Error loading the dependency list resource: " + getTestingFrameworkBundlesConfiguration();
        log.error(error, e);
        throw new IllegalArgumentException(error, e);
    }
 
    if (log.isTraceEnabled()) {
        log.trace("Maven artifacts " + artifacts);
    }
  
    // Filter and get versions
    Iterator<MavenArtifact> iter = artifacts.iterator();
    while (iter.hasNext()) {
        MavenArtifact artifact = iter.next();
        if (artifact.getGroupId().equals(SPRING_GROUP_ID)
                && artifact.getArtifactId().equals(SPRING_CONTEXT_ID)) {
            springBundledVersion = artifact.getVersion();
        } else if (artifact.getGroupId().equals(SPRING_OSGI_GROUP_ID)) {
            if (artifact.getArtifactId().equals(SPRING_OSGI_CORE_ID)) {
                springOsgiVersion = artifact.getVersion();
            }
        } else if (artifact.getGroupId().equals(ECLIPSE_OSGI)
                && artifact.getArtifactId().equals(ECLIPSE_OSGI)) {
            // filter out this since it is started by the framework specification in
            // the POM.
            iter.remove();
        } else if (artifact.getGroupId().equals(BACKPORT_GROUP_ID)
                && JdkVersion.isAtLeastJava15()) {
            // Filter out backport if Java is 1.5 or higher. 
            iter.remove();
        }
    }
 
    // Copy the filtered artifacts to the bundle array that we need
    // to return from this method.
    String[] bundles = new String[artifacts.size()];
    iter = artifacts.iterator();
    for (int i = 0; iter.hasNext(); ++i) {
        MavenArtifact artifact = iter.next();
 
        bundles[i] = artifact.getGroupId() + "," + artifact.getArtifactId() + ","
                + artifact.getVersion();
    }

    // pass properties to test instance running inside OSGi space
    System.getProperties().put(SPRING_OSGI_VERSION_PROP_KEY, springOsgiVersion);
    System.getProperties().put(SPRING_VERSION_PROP_KEY, springBundledVersion);

    // sort the array (as the dependency list can be any order)
    // Is this really required??
    bundles = StringUtils.sortStringArray(bundles);
    if (log.isDebugEnabled()) {
        log.debug("Default framework bundles :" + ObjectUtils.nullSafeToString(bundles));
    }
 
    return bundles;
} 

The constants used in the above methods are:

/** The path to the resource containing the default bundles to load. */
protected static final String DEFAULT_DPENDENCIES_LIST_FILENAME = "target/dependencies.list";

/** Maven groupId for Spring. */
public static final String SPRING_GROUP_ID = "org.springframework";
/** Maven groupId for Spring OSGi. */
public static final String SPRING_OSGI_GROUP_ID = SPRING_GROUP_ID + ".osgi";
/** Maven artifactId for the Spring context bundle. */
public static final String SPRING_CONTEXT_ID = SPRING_GROUP_ID + ".context";
/** Maven artifactId for the Spring OSGi core bundle. */
public static final String SPRING_OSGI_CORE_ID = SPRING_OSGI_GROUP_ID + ".core";
/** Maven artifactId for the Spring OSGi Test bundle. */
public static final String SPRING_OSGI_TEST_ARTIFACT_ID = SPRING_OSGI_GROUP_ID + ".test";
/** Maven groupId and artifactId for the eclipse framework. */
public static final String ECLIPSE_OSGI = "org.eclipse.osgi";
/** Maven groupId for the backport software library. */
public static final String BACKPORT_GROUP_ID = "edu.emory.mathcs.backport";

/** Store the Spring OSGi version in a system property for the test classes. */
private static final String SPRING_OSGI_VERSION_PROP_KEY = "spring.osgi.version";
/** Store the Spring version in a system property for the test classes. */
private static final String SPRING_VERSION_PROP_KEY = "spring.version";

You also have to override two other methods to return the versions of Spring and Spring-DM that are used. These versions will be discovered from the list of dependencies.

/**
 * Returns the version of the Spring-DM bundles installed by the testing
 * framework.
 * 
 * @return Spring-DM bundles version.
 */
@Override
protected String getSpringDMVersion() {
    if (springOsgiVersion == null) {
        springOsgiVersion = System.getProperty(SPRING_OSGI_VERSION_PROP_KEY);
    }
    return springOsgiVersion;
}

/**
 * Returns the version of the Spring bundles installed by the testing
 * framework.
 * 
 * @return Spring framework dependency version.
 */
@Override
protected String getSpringVersion() {
    if (springBundledVersion == null) {
        springBundledVersion = System.getProperty(SPRING_VERSION_PROP_KEY);
    }
    return springBundledVersion;
}

The getTestFrameworkBundlesNames() method above uses two classes to parse the dependency list:

package com.infor.springframework.osgi.test.internal;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.Reader;
import java.util.ArrayList;
import java.util.List;

import org.springframework.core.io.Resource;

/**
 * Parse the dependency list produced by the Maven dependency plugin
 * and create Maven artifact definitions for each dependency specified.
 * 
 * @author dlaidlaw
 *
 */
public final class MavenDependencyListParser {

    /**
     * Private to prevent instantiation.
     */
    private MavenDependencyListParser() {
        super();
    }

    /**
     * Parse the Maven dependencies list specified as a Spring resource.
     * @param resource the Spring resource pointing to the dependencies list.
     * @return the artifacts from the dependency list.
     * @throws IOException if the resource cannot be read.
     */
    public static List<MavenArtifact> parseDependencies(final Resource resource)
    throws IOException {
        return parseDependencies(new InputStreamReader(resource.getInputStream()));
    }
 
    /**
     * Parse the Maven dependencies list specified as a {@link Reader}.
     * @param reader the reader to use for reading the dependencies list.
     * @return the artifacts from the dependency list.
     * @throws IOException if the resource cannot be read.
     */
    public static List<MavenArtifact> parseDependencies(final Reader reader)
    throws IOException {
        BufferedReader in = new BufferedReader(reader);
        ArrayList<MavenArtifact> artifacts = new ArrayList<MavenArtifact>();
        String line = in.readLine();
        while (line != null) {
            if (isSpecLine(line)) {
                artifacts.add(MavenArtifact.parse(line));
            }
            line = in.readLine();
        }
        return artifacts;
    }
 
    /**
     * A spec line is a line specifying a Maven artifact. For it to be valid
     * there must be at least three colon characters on the line.
     * @param line the line to check.
     * @return true if it is a spec line.
     */
    private static boolean isSpecLine(final String line) {
        int i = line.indexOf(':');
        if (i > 0) {
            // check for a second colon.
            i = line.indexOf(':', i + 1);
            if (i > 0) {
                // check for a third colon
                return (line.indexOf(':', i + 1) >= 0);
            }
        }
        return false;
    }
}
package com.infor.springframework.osgi.test.internal;

/**
 * A maven artifact information container. Holds the maven artifact
 * info. Once an instance of this object is created it cannot be changed.
 * @author dlaidlaw
 *
 */
public class MavenArtifact {
    /** The artifact groupId. */
    private String groupId;
    /** The artifact artifactId. */
    private String artifactId;
    /** The artifact type (jar, war, zip, etc). */
    private String type;
    /** The artifact version. */
    private String version;
    /** The artifact scope. */
    private String scope;

    /** The string value of this object, cached so it does not need to be
     * continuously recalculated.
     */
    private transient String stringValue;

    /**
     * Default constructor. All values are null.
     */
    private MavenArtifact() {
        super();
    }

    /**
     * Full constructor, All values are provided.
     * @param groupId the artifact groupId.
     * @param artifactId the artifact ID.
     * @param type the artifact type (jar, war, zip, etc).
     * @param version the artifact version.
     * @param scope the artifact scope (compile, test, etc);
     */
    public MavenArtifact(final String groupId, final String artifactId,
            final String type, final String version, final String scope) {
        super();
        this.groupId = groupId;
        this.artifactId = artifactId;
        this.type = type;
        this.version = version;
        this.scope = scope;
    }
 
    /**
     * Parse a maven artifact specification as output by the Maven dependency plugin
     * resolve goal.
     * @param spec the specification string formatted as
     *   groupId:artifactId:type:version:scope:other.
     * @return the artifact.
     * @see http://maven.apache.org/plugins/maven-dependency-plugin/resolve-mojo.html
     * @throws IllegalArgumentException if the specification string does not contain
     *   the required parts with colon separators.
     */
    public static MavenArtifact parse(final String spec) {
        String[] parts = spec.trim().split(":");
        // CHECKSTYLE:OFF the number four is not magic.
        if (parts.length < 4) {
            // CHECKSTYLE:ON
            throw new IllegalArgumentException(
                    "The specification must contain at least 5 parts separated by a colon (:)."
                    + " The parts are: groupId:artifactId:type:version:scope:other");
        }

        MavenArtifact ma = new MavenArtifact();
        if (parts[0].length() > 0) {
            ma.groupId = parts[0];
        }
        if (parts[1].length() > 0) {
            ma.artifactId = parts[1];
        }
        if (parts[2].length() > 0) {
            ma.type = parts[2];
        }
        // CHECKSTYLE:OFF the numbers three and four are not magic.
        if (parts[3].length() > 0) {
            ma.version = parts[3];
        }
        if (parts.length >= 4 && parts[4].length() > 0) {
            ma.scope = parts[4];
        }
        // CHECKSTYLE:ON

        return ma;
    }
 
    /**
     * Output the values separated by colons exactly as the 
     * parse method would like to see them. Null values are
     * output as the empty string.
     * @return the string representation of this object.
     */
    @Override
    public String toString() {
        if (stringValue == null) {
            stringValue = asString(groupId) + ":" + asString(artifactId) + ":" 
                + asString(type) + ":" + asString(version)
                + ":" + asString(scope);
        }
        return stringValue;
    }

    /**
     * Return "" if the argument is null.
     * @param s the string to output.
     * @return "" if s is null, or s otherwise.
     */
    private String asString(final String s) {
        if (s == null) {
            return "";
        }
        return s;
    }

    /**
     * Check if the other object is an instance of this
     * class and that all its parts equal the parts in
     * this instance.
     * @param other the object to test against this object.
     * @return true if the other object is an instance of
     *   this class and all its parts equals the parts
     *   of this class.
     */
    @Override
    public boolean equals(final Object other) {
        if ((other == null) || !(other instanceof MavenArtifact)) {
            return false;
        }
        MavenArtifact o = (MavenArtifact) other;
        return (isEqual(this.groupId, o.getGroupId())
                && isEqual(this.artifactId, o.getArtifactId())
                && isEqual(this.type, o.getType())
                && isEqual(this.version, o.getVersion())
                && isEqual(this.scope, o.getScope()));
    }
 
    /**
     * Compare this object to the other object, returning true if the
     * other object's groupId and artifactId are equal to this object's
     * groupId and artifactId.
     * @param other the object to compare to this one.
     * @return true if the other equals this object on the two
     *   id properties.
     */
    public boolean idsEqual(final MavenArtifact other) {
        if (other == null) {
            return false;
        }
        return isEqual(this.groupId, other.getGroupId())
                && isEqual(this.artifactId, other.getArtifactId());
    }

    /**
     * Output the hash code for this object.
     * @return the hashCode calculated as the hashCode of
     *   the string returned by the toString method.
     * @see #toString()
     */
    @Override
    public int hashCode() {
        return toString().hashCode();
    }

    /**
     * Both strings are null or they are equal.
     * @param a one string.
     * @param b the other string.
     * @return true if both are null, or both are equal.
     */
    private boolean isEqual(final String a, final String b) {
        if (a == null) {
            return b == null;
        }
        if (b == null) {
            return false;
        }
        return a.equals(b);
     }

    /**
     * Get the groupId.
     * @return the groupId.
     */
    public String getGroupId() {
        return groupId;
    }

    /**
     * Get the artifactId.
     * @return the artifactId.
     */
    public String getArtifactId() {
        return artifactId;
    }

    /**
     * Get the artifact type (jar, war, zip, etc).
     * @return the type.
     */
    public String getType() {
        return type;
    }

    /**
     * Get the version.
     * @return the version.
     */
    public String getVersion() {
        return version;
    }

    /**
     * Get the scope.
     * @return the scope.
     */
    public String getScope() {
        return scope;
    } 
}

The POM

Your Maven project for the above code will require a POM file. The dependencies in the new test project will also become transient dependencies of any integration test projects that reference this project, and therefore the correct set of bundles will be loaded into the OSGi container. My POM for this project includes these depedencies:

<description>A testing utility bundle that allows the user to set which
    version of spring and spring-dm is being used in the test.
</description>

<properties>
    <version.eclipse.osgi>3.5.0.v20081201-1815</version.eclipse.osgi>
    <version.eclipse.osgi.services>3.1.200.v20071203</version.eclipse.osgi.services>
    <version.equinox.cm>1.0.0.v20080929-1800</version.equinox.cm>
    <version.equinox.event>1.1.0.v20080225</version.equinox.event>
    <version.junit>3.8.2</version.junit>
    <version.spring>2.5.6</version.spring>
    <version.spring.dm>1.2.0</version.spring.dm>
    <version.objectweb.asm>2.2.3</version.objectweb.asm>
    <version.cglib>2.1.3</version.cglib>
    <version.pax.logging>1.3.0</version.pax.logging>
</properties>

<dependencies>
    <dependency>
        <groupId>org.springframework.osgi</groupId>
        <artifactId>org.springframework.osgi.test</artifactId>
        <version>${version.spring.dm}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.osgi</groupId>
        <artifactId>org.springframework.osgi.core</artifactId>
        <version>${version.spring.dm}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.osgi</groupId>
        <artifactId>org.springframework.osgi.io</artifactId>
        <version>${version.spring.dm}</version>
       <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.osgi</groupId>
        <artifactId>org.springframework.osgi.extender</artifactId>
        <version>${version.spring.dm}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework.osgi</groupId>
        <artifactId>org.springframework.osgi.extensions.annotation</artifactId>
        <version>${version.spring.dm}</version>
        <scope>test</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>org.springframework.test</artifactId>
        <version>${version.spring}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.springframework</groupId>
        <artifactId>org.springframework.context</artifactId>
        <version>${version.spring}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.objectweb.asm</groupId>
        <artifactId>com.springsource.org.objectweb.asm</artifactId>
        <version>${version.objectweb.asm}</version>
    </dependency>
    <dependency>
        <groupId>net.sourceforge.cglib</groupId>
        <artifactId>com.springsource.net.sf.cglib</artifactId>
        <version>${version.cglib}</version>
    </dependency>
    <dependency>
        <groupId>org.eclipse.osgi</groupId>
        <artifactId>org.eclipse.osgi</artifactId>
        <version>${version.eclipse.osgi}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.osgi</groupId>
        <artifactId>org.eclipse.osgi.services</artifactId>
        <version>${version.eclipse.osgi.services}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.equinox</groupId>
        <artifactId>org.eclipse.equinox.cm</artifactId>
        <version>${version.equinox.cm}</version>
        <scope>compile</scope>
    </dependency>
    <dependency>
        <groupId>org.eclipse.equinox</groupId>
        <artifactId>org.eclipse.equinox.event</artifactId>
        <version>${version.equinox.event}</version>
        <scope>compile</scope>
    </dependency>
    <!-- Logging -->
    <dependency>
        <groupId>org.ops4j.pax.logging</groupId>
        <artifactId>pax-logging-api</artifactId>
        <version>${version.pax.logging}</version>
        <scope>provided</scope>
    </dependency>
    <dependency>
        <groupId>org.ops4j.pax.logging</groupId>
        <artifactId>pax-logging-service</artifactId>
        <version>${version.pax.logging}</version>
        <scope>provided</scope>
        <exclusions>
            <exclusion>
                <artifactId>log4j</artifactId>
                <groupId>log4j</groupId>
            </exclusion>
        </exclusions>
    </dependency>
    <!-- JUnit -->
    <dependency>
        <groupId>org.junit</groupId>
        <artifactId>com.springsource.junit</artifactId>
        <version>${version.junit}</version>
        <scope>compile</scope>
    </dependency>
</dependencies>

Note that I changed the set of bundles loaded from what Spring would load. I used pax-logging instead of SLF4J. At this time the Spring integration test code does not work with JUnit 4, so don't be tempted to update that version until Spring changes the base test framework to accommodate the newer versions.

Summary

Now all you need to do is reference your new project containing the code above in your integration tests and set your POM dependencies to the bundles that are required for the test. All the transient dependencies will be discovered and everything will load into the integration test container.

Note that I used the Eclipse Equinox OSGi container and hard coded that into the code above where the platform bundle is filtered out of the bundles loaded (because it is already loaded by the Spring test framework). It is easy to tweek that section to detect and remove the appropriate bundle if it is present.

No comments:

Post a Comment