Building eXist-db

(1Q18)


This article will tell you how to build a Java .war or .jar file for eXist-db.

Building a .war file from the Standard Distribution

If you would like to deploy eXist-db into an existing servlet engine (e.g. tomcat), you will need a .war file. We no longer distribute a .war build, but it is easy to create one from the standard distribution by repackaging (not building) it:

  1. Install the standard distribution

    Get a standard distribution from eXist-db download page. Install it into a directory of your choice. Make sure you include the source package.

  2. Generate the WAR

    Open a command line prompt and change to the directory where you just installed eXist-db. Call

    build.sh dist-war

    or

    build.bat dist-war

    once. This will generate a .war archive into directory dist.

Note that generating a .war archive does only repackage the files in the distribution. So if the distribution worked properly for you, the .war should as well. You don't need to recompile eXist-db or sign any jars.

Building eXist-db from GitHub

Building eXist-db is quite easy and requires 3 simple steps:

  1. Clone from Github

    The GitHub URL for the develop branche is:

    git@github.com:eXist-db/exist.git

    Please follow the instructions on our GitHub page page ( README.md section)

  2. Set JAVA_HOME

    Before starting the build, your JAVA_HOME environment variable should be set to point to the root directory of the JAVA JDK (note: JDK, not JRE!). Open a console or a DOS box on windows and type:

    set JAVA_HOME=c:\path\to\jdk

    on Windows or

    export JAVA_HOME=/path/to/jdk

    on Unix.

  3. Build eXist

    Change into the eXist-db directory you just checked out and call

    build.bat

    on Windows or on Unix:

    build.sh

Using the Build System

You can rebuild all eXist-db packages from the standard distribution or the GitHub repository. You can even create a new distribution from the one you installed. For example, assume you installed eXist using the installer GUI, but you need to have an exist.war package for deployment within a different servlet engine. Just call the main build script (build.sh or build.bat) and pass it the target dist-war:

build.sh dist-war

This will create a fresh .war archive in the dist directory.

To be able to use the build system, you need to include the "source" module when selecting installable packages in the installer GUI.

eXist-db relies on Ant for all compile and build tasks. Ant is included in the distribution (directory tools/ant). As already mentioned above, there are two shell scripts to start Ant: build.sh or build.bat. The shell scripts will take care to create a proper command line to launch Java with the Ant version included in the distribution.

Calling Ant without further argument will compile the sources and create the main .jar files (exist.jar, exist-optional.jar, start.jar).

build.sh -plists the available build targets. Passing one of these as argument to ant will call execute the corresponding target. The main targets are:

all

Compile the sources and create the main .jar files (exist.jar, exist-optional.jar, start.jar).

benchmark

Performs the benchmark tests for eXist-db.

clean

Cleans the files and directories generated by a previous build. Data is preserved.

clean-all

Cleanup deeper.

clean-conf

Cleanup config file.

clean-default-data-dir

Remove contents in default data directory.

compile

Compiles the source code.

compile-aspectj

Compiles the aspects.

dist-tgz

Packages eXist-db into a .tgz file.

dist-war

Packages eXist-db into a .war file for deployment within a servlet engine.

dist-zip

Packages eXist-db into a .zip file.

download-additional-jars

Download optional third-party .jar files. These are currently: saxon. These are downloaded into lib/user/.

download-icu4j

Download the optional icu4j library.

download-xars

Download xars to include in installer.

download-xqts

Download XQTS file.

download-xslts

Download XSLTS file.

extension-modules

Build the extensions directory. Edit the file extensions/build.properties enable extensions and modules.

installer

Creates a new installer package based on IzPack.

installer-exe

Create .exe from installer

jar

The default: compiles the source and creates eXist's main .jar files.

javadocs

Creates the API docs into webapp/api.

jdepend

JDepend traverses Java class file directories and generates design quality metrics for each Java package.

jetty-keygen

Generate keystore for jetty.

jnlp-clean

jnlp-keygen

Generate keystore for signing .jar files.

jnlp-pack200

Pack all .jar files.

jnlp-sign-all

This signs all of the .jar files, so that the webstart of the eXist-db client Java application will launch without any signing errors.

jnlp-sign-core

Sign all CORE .jar files.

jnlp-sign-exist

Sign all EXIST .jar files.

jnlp-unsign-all

This unsigns all of the .jar files.

rebuild

Performs a clean and the all.

samples

Builds the examples and packs them into examples.jar.

snapshot-installer

Creates a snapshot installer.

test

Runs all tests in the test suite and outputs the HTML result into test/junit/html. This may take several minutes.

xquery

Regenerate the XQuery parser code with ANTLR (only required if you change src/org/exist/xquery/parser/XQuery.g).

xars

Scan apps directory and include all .xar files into installer.

You might want to set some custom settings (in particular your HTTP proxy when external libraries are required) before running build.sh or build.bat. See build.properties.

Signing .jar files for webstart

The database can be easily accessed with the Java client, when the client is launched by clicking the Java Webstart "Launch" button on the right side of this page. Java webstart explicilty requires that all application code, the .jar files, are signed using a security certificate. Out of the box, when eXist-db is installed using the installer, all relevant .jar files have already been signed, so webstart can be used directly. It is necessary however to (re-)sign the .jar files yourself if you want to (re-)build the eXist-db java code. This chapter explains how to do this.

Quickstart

To get started quickly, just execute the following command:

./build.sh -f build/scripts/jarsigner.xml

This command sequentially creates the key.store file and signs a selection of .jar files:

./build.sh -f build/scripts/jarsigner.xml 
Starting Ant...

Buildfile: ..../eXist-db/build/scripts/jarsigner.xml

jnlp-prepare:

jnlp-keygen:
   [genkey] Generating Key for exist

jnlp-sign-exist:
  [signjar] Signing JAR: ..../eXist-db/exist-fluent.jar to ..../eXist-db/exist-fluent.jar as exist
  [signjar] Signing JAR: ..../eXist-db/exist-optional.jar to ..../eXist-db/exist-optional.jar as exist
  [signjar] Signing JAR: ..../eXist-db/exist.jar to ..../eXist-db/exist.jar as exist
  [signjar] Signing JAR: ..../eXist-db/start.jar to ..../eXist-db/start.jar as exist

jnlp-sign-core:
  [signjar] Signing JAR: ..../eXist-db/lib/core/antlr-2.7.7.jar to ..../eXist-db/lib/core/antlr-2.7.7.jar as exist
  [signjar] Signing JAR: ..../eXist-db/lib/core/commons-collections-3.2.1.jar to ..../eXist-db/lib/core/commons-collections-3.2.1.jar as exist
  ....
  [signjar] Signing JAR: ..../eXist-db/lib/core/xmlrpc-server-3.1.3.jar to ..../eXist-db/lib/core/xmlrpc-server-3.1.3.jar as exist

Note that this command needs to be executed just once. In subsequent builds newly generated .jar files are signed automatically.

Ant targets

An overview of all available ant targets can be retrieved by executing the following command:

./build.sh -projecthelp -f build/scripts/jarsigner.xml

The following ant targets are available for signing .jar files:

jnlp-all

Create keystore file and sign all EXIST and CORE .jar files. (Default task)

jnlp-keygen

Generate a new keystore file if not present.

jnlp-clean

Delete the keystore file.

jnlp-sign-core

Sign all CORE .jar files in lib/core.

jnlp-sign-exist

Sign all EXIST .jar files, e.g. exist.jar exist-XX.jar and start.jar

jnlp-unsign-all

Unsign all .jar files.

The target jnlp-keygen generates a new certificate file key.store. If this file is present the EXIST .jar files are signed during each following build. The process of signing .jar files can be stopped by removing this keystore file.

The ant script contains two additional targets: jetty-keygen which generates a keystore file for the jetty server and jnlp-pack200 which processes the signed .jar files with pack200 technology. More about the latter in the next section.

Reducing .jar sizes with pack200 compression

Pack200 is a compression technology designed for compressing .jar files. It works most efficiently on Java class files. Pack200 can reduce the size of a .jar file up to 60% resulting into a significant reduction of the amount of downloaded webstart data.

The webstart .jar files can be compressed with pack200 with the following command:

./build.sh -f build/scripts/jarsigner.xml jnlp-pack200

This command consecutively performs the following steps for all .jar files:

  • unsign

  • repack (normalization)

  • sign

  • compress with pack200

  • compress with gzip

./build.sh -f build/scripts/jarsigner.xml jnlp-pack200
                    Starting Ant...
                    
                    Buildfile: .../eXist-db/build/scripts/jarsigner.xml
                    
                    jnlp-pack200:
                    
                    jnlp-unsign-all:
                    [unsignjar] Unsigning exist-fluent.jar
                    [unsignjar] Unsigning exist-optional.jar
                    [unsignjar] Unsigning exist.jar
                    [unsignjar] Unsigning start.jar
                    [unsignjar] Unsigning antlr-2.7.7.jar
                    ...
                    [unsignjar] Unsigning xmlrpc-server-3.1.3.jar
                    [repack] Normalizing exist-fluent.jar
                    [repack] Normalizing exist-optional.jar
                    [repack] Normalizing exist.jar
                    [repack] Normalizing start.jar
                    [repack] Normalizing antlr-2.7.7.jar
                    ...
                    [repack] Normalizing xmlrpc-server-3.1.3.jar
                    
                    jnlp-prepare:
                    
                    jnlp-keygen:
                    
                    jnlp-sign-exist:
                    [signjar] Signing JAR: .../eXist-db/exist-fluent.jar to .../eXist-db/exist-fluent.jar as exist
                    [signjar] Signing JAR: .../eXist-db/exist-optional.jar to .../eXist-db/exist-optional.jar as exist
                    [signjar] Signing JAR: .../eXist-db/exist.jar to .../eXist-db/exist.jar as exist
                    [signjar] Signing JAR: .../eXist-db/start.jar to .../eXist-db/start.jar as exist
                    
                    jnlp-prepare:
                    
                    jnlp-keygen:
                    
                    jnlp-sign-core:
                    [signjar] Signing JAR: .../eXist-db/lib/core/antlr-2.7.7.jar to .../eXist-db/lib/core/antlr-2.7.7.jar as exist
                    ...
                    [signjar] Signing JAR: .../eXist-db/lib/core/xmlrpc-server-3.1.3.jar to .../eXist-db/lib/core/xmlrpc-server-3.1.3.jar as exist
                    [pack] Packing exist-fluent.jar
                    [pack] Packing exist-optional.jar
                    [pack] Packing exist.jar
                    [pack] Packing start.jar
                    [pack] Packing antlr-2.7.7.jar
                    ...
                    [pack] Packing xmlrpc-server-3.1.3.jar
                    
                    BUILD SUCCESSFUL
                    Total time: 1 minute 0 seconds

References

Notes

  • Sometimes when the .jar files are signed, eXist-db does not start anymore. In this situation unsign the .jar files first. Calling target 'jnlp-pack200' does always unsign all .jar files as first step.

  • The pack200 target is not executed automatically. If eXist-db code is changed and recompiled, you might want to execute the jnlp-pack200 target again.