Modular runtime images with Maven

The diez program I talked about in a previous post ended up being just a command-line program. Since all my new Java projects are modular by default, it includes a module-info.java file that declares the required modules for the program to run (in this case, the ez.vcard module).

But since the program is supposed to be distributed to third parties, I decided to package it as an über-jar. The relevant configuration in the POM file is as follows:

<project>
  ...
  <dependencies>
    ...
    <dependency>
      <groupId>com.googlecode.ez-vcard</groupId>
      <artifactId>ez-vcard</artifactId>
      <version>0.10.5</version>
      <exclusions>
        <exclusion>
          <groupId>org.jsoup</groupId>
          <artifactId>jsoup</artifactId>
        </exclusion>
        <exclusion>
          <groupId>org.freemaker</groupId>
          <artifactId>freemaker</artifactId>
        </exclusion>
        <exclusion>
          <groupId>com.fasterxml.jackson.core</groupId>
          <artifactId>jackson-core</artifactId>
        </exclusion>
        <exclusion>
          <groupId>commons-codec</groupId>
          <artifactId>commons-codec</artifactId>
        </exclusion>
      </exclusions>
    </dependency>
  </dependencies>
  ...
  <build>
    <plugins>
      <!-- Create fat JAR -->
      <plugin>
        <artifactId>maven-shade-plugin</artifactId>
        <version>3.2.1</version>
        <executions>
          <execution>
            <phase>package</phase>
            <goals>
              <goal>shade</goal>
            </goals>
            <configuration>
              <transformers>
                <transformer implementation="...">
                  <manifestEntries>
                    <Main-Class>mx.amvc.diez.Main</Main-Class>
                  </manifestEntries>
                </transformer>
              </transformers>
              <artifactSet>
                <includes>
                  <include>com.googlecode.ez-vcard:ez-vcard</include>
                  <include>com.github.mangstadt:vinnie</include>
                </includes>
              </artifactSet>
            </configuration>
          </execution>
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

As you can see from the snippets above, the only (direct) dependency is the com.googlecode.ez-vcard:ez-vcard artifact; however, when creating the fat JAR, we also need to include the com.github.mangstadt:vinnie artifact, because it is needed by ez-vcard itself.

Now, when building the package, the maven-shade-plugin emits a warning:

[WARNING] Discovered module-info.class. Shading will break its strong
encapsulation.

This makes sense, since über-jars go against the philosophy and motivation of the Java 9 Module System.

But I want to be able to create a custom runtime image that includes only my module, the two dependency modules, and at most any other JVM base modules the dependencies may need.

And I want to do it using mvn package, because I want to keep my build process as simple as possible. I was successful at doing it by hand:

Compile my project classes using javac.
Package my project classes using jar.
Copy the two dependency JARs from the local Maven repository.
Generate each dependency's module-info.java and inject it in the JAR.
Use jlink to generate the custom runtime image from the three modular JARs.

But this is surely a more complicated process.

`maven-jlink-plugin`

My first approach was using maven-jlink-plugin. However, this didn't work because of a couple of major issues:

The plugin is still in alpha phase, so there is a high chance I will need to modify the code later.
The plugin cannot deal with dependent modules at all.

All this lead me to try another approach. After some research, I ended up using the Moditect plugin.

Moditect

Using this plugin is really easy, although some things are not clear in the documentation. It lets you do three different things:

Make your JAR modular (even if your project lacks a module-info.java file).
Make your dependencies modular; this includes repackaging plain old JARs as modules.
Create a runtime image out of the modules from the two steps above.

This is all I needed!

The first thing to do is remove the maven-shade-plugin configuration from the POM, and replace it with this:

<project>
  ...
  <build>
    ...
    <plugins>
      <!-- Create modular JAR -->
      <plugin>
        <groupId>org.moditect</groupId>
        <artifactId>moditect-maven-plugin</artifactId>
        <version>1.0.0.Beta2</version>
        <executions>
          ...
        </executions>
      </plugin>
    </plugins>
  </build>
</project>

Each one of the three steps above will be configured as a separate execution.

Make your JAR modular

This first step is one of those things that is not clear in the plugin documentation.

Although my project already has module-info.java file, you still need to configure it in the POM. There is a little bit of duplication that, if removed, could significantly improve the plugin usage.

The first execution will be configured to run with the add-module-info goal of the package phase:

  <!-- Generate project module -->
  <execution>
    <id>generate-project-module</id>
    <phase>package</phase>
    <goals>
      <goal>add-module-info</goal>
    </goals>
    <configuration>
      <jvmVersion>${properties.maven.compiler.target}</jvmVersion>
      <module>
        <moduleInfoFile>src/main/java/module-info.java</moduleInfoFile>
      </module>
    </configuration>
  </execution>

The interesting bit here is the moduleInfoFile element, with which we are telling the plugin that we already have a module-info.java file and should use it as-is.

Make your dependencies modular

The second step is to inject a module-info.java file to each dependency JAR. This is done with an execution configured to run with the add-module-info goal of the generate-resources phase:

  <!-- Add module-info.java to the dependencies -->
  <execution>
    <id>add-module-infos</id>
    <phase>generate-resources</phase>
    <goals>
      <goal>add-module-info</goal>
    </goals>
    <configuration>
      <outputDirectory>${project.build.directory}/modules</outputDirectory>
      <modules>
        <module>
          <artifact>
            <groupId>com.googlecode.ez-vcard</groupId>
            <artifactId>ez-vcard</artifactId>
            <version>0.10.5</version>
          </artifact>
          <moduleInfoSource>
            module ez.vcard {
                exports ezvcard;
                exports ezvcard.io.chain;
                exports ezvcard.property;

                requires vinnie;
                requires java.xml;
            }
          </moduleInfoSource>
        </module>
        <module>
          <artifact>
            <groupId>com.github.mangstadt</groupId>
            <artifactId>vinnie</artifactId>
            <version>2.0.2</version>
          </artifact>
          <moduleInfo>
            <name>vinnie</name>
            <exports>
              *;
            </exports>
          </moduleInfo>
        </module>
      </modules>
    </configuration>
  </execution>

This configuration is (more or less) straighforward: first, we define where we want to export our modularized dependencies; then we include a module entry for each dependency JAR.

Each module includes two elements:

The dependency GAV in the artifact element.
The moduleInfo configuration.

The configuration for the vinnie module is simple: a name so the Java Module System doesn't have to guess it, and an exports clause (in this case, export every package).

However, the configuration of ez.vcard is more complicated. I'm not really sure why I had to put everything in a moduleInfoSource element instead of a simpler moduleInfo element (this is another thing that could be improved in the plugin documentation), but let's see the process I had to follow to arrive to this configuration.

We can start with the following execution and proceed from there:

<module>
  <artifact>
    <groupId>com.googlecode.ez-vcard</groupId>
    <artifactId>ez-vcard</artifactId>
    <version>0.10.5</version>
  </artifact>
  <moduleInfo>
    <name>ez.vcard</name>
    <exports>
      ezvcard;
    </exports>
    <requires>
      vinnie;
    </requires>
  </moduleInfo>
</module>

This throws the following error:

[ERROR] Error: Module org.jsoup not found, required by ez.vcard

Please note that the org.jsoup artifact is explicitly excluded from the dependency configuration, since I don't use it at all in my code:

<project>
  ...
  <dependencies>
    ...
    <dependency>
      <groupId>com.googlecode.ez-vcard</groupId>
      <artifactId>ez-vcard</artifactId>
      <version>0.10.5</version>
      <exclusions>
        <exclusion>
          <groupId>org.jsoup</groupId>
          <artifactId>jsoup</artifactId>
        </exclusion>
        ...
      </exclusions>
    </dependency>
  </dependencies>
  ...
</project>

I'm not sure what is going on here, but it is solved if we use a moduleInfoSource instead:

<module>
  <artifact>
    <groupId>com.googlecode.ez-vcard</groupId>
    <artifactId>ez-vcard</artifactId>
    <version>0.10.5</version>
  </artifact>
  <moduleInfoSource>
    module ez.vcard {
        exports ezvcard;
        requires vinnie;
    }
  </moduleInfoSource>
</module>

The build succeeds in this case!

The astute reader will notice that we have missing exports and requires; we will talk about that below.

Create a custom runtime image

The third and final step is to create a custom runtime image with just the necessary modules. The configuration for that is achieved by the create-runtime-image goal of the package phase:

  <!-- Generate runtime image -->
  <execution>
    <id>create-image</id>
    <phase>package</phase>
    <goals>
      <goal>create-runtime-image</goal>
    </goals>
    <configuration>
      <modulePath>
        <path>${project.build.directory}/modules</path>
      </modulePath>
      <modules>
        <module>mx.amvc.diez</module>
      </modules>
      <launcher>
        <name>diez</name>
        <module>mx.amvc.diez/mx.amvc.diez.Main</module>
      </launcher>
      <outputDirectory>${project.build.directory}/image</outputDirectory>
    </configuration>
  </execution>

The configuration is straightforward, too:

The modulePath tells the plugin where to pull the modular JARs from. This should match the outputDirectory of the add-module-infos execution.
The modules element declares which modules we want to include in our custom image. Please notice that the dependencies are not included here, since those are automatically resolved for us. Nice!
The launcher element is optional, but convenient. This matches the syntax of the jlink command's --launcher option.
Finally, outputDirectory specifies the location of the custom image.

With this in place, we can build the custom image and verify the modules in it:

$ mvn clean package
$ ./target/image/bin/java --list-modules
ez.vcard@0.10.5
java.base@11.0.4
mx.amvc.diez@1.0
vinnie@2.0.2

Great! The custom image includes our module, the two dependent modules, and the java.base required module.

Now, we can run the program:

$ ./target/image/bin/diez contacts.vcf output.vcf
Exception in thread "main" java.lang.IllegalAccessError:
    class mx.amvc.diez.ContactProcessor (in module mx.amvc.diez) cannot access
    class ezvcard.io.chain.ChainingTextParser (in module ez.vcard) because
    module ez.vcard does not export ezvcard.io.chain to module mx.amvc.diez
    ...

Oh!

Fixing the `ez.vcard` module configuration

Our code only uses the ezvcard package but, internally, such package requires ezvcard.io.chain. After adding that exports clause to ez.vcard's moduleInfoSource configuration, we get the following:

$ mvn clean package
$ ./target/image/bin/diez contacts.vcf output.vcf
Exception in thread "main" java.lang.NoClassDefFoundError:
    javax/xml/namespace/QName
    ...
Caused by: java.lang.ClassNotFoundException: javax.xml.namespace.QName
    ...

OK, it looks like we also need to require the java.xml module. Let's try again:

$mvn clean package
$ ./target/image/bin/java --list-modules
ez.vcard@0.10.5
java.base@11.0.4
java.xml@11.0.4
mx.amvc.diez@1.0
vinnie@2.0.2

As you can see, now the java.xml module is included in the custom image. Let's run the program:

$ ./target/image/bin/diez contacts.vcf output.vcf
Exception in thread "main" java.lang.IllegalAccessError:
    class mx.amvc.diez.ContactProcessor (in module mx.amvc.diez) cannot access
    class ezvcard.property.FormattedName (in module ez.vcard) because
    module ez.vcard does not export ezvcard.property to module mx.amvc.diez
    ...

Another package to export. Let's add it and try one more time:

$ mvn clean package
$ ./target/image/bin/diez contacts.vcf output.vcf
Frodo:
 - +52  55 1111 1111 --> 5511111111
Sam:
 - +52-155-1111-2222 --> 5511112222
Merry:
 - +1-416-111-3333 --> +14161113333
Pippin:
 - 5511114444 --> 5511114444
Aragorn, Son of Arathorn:
 - 1111.5555 --> 5511115555
Legolas:
 - 01-81-1111-6666 --> 8111116666
Gimli:
 - 044 55-1111 7777 --> 5511117777
Boromir:
 - 045-55-11118888 --> 5511118888
Sauron:
 - FooBarBazQuux --> 
Saruman:
 - +213 --> +213

Finally, it works!

Optimizing the runtime image in Debian systems

Custom runtime images generated with the Debian OpenJDK package are really large:

$ du -sh target/image/
407M    target/image/

We can remove all the extra debug symbols in the image with strip:

strip -p --strip-unneeded target/image/lib/server/libjvm.so

The final size is much smaller:

$ du -sh target/image/
58M target/image/

maven-jlink-plugin