While this setup is specifically for Travis-CI you will possibly find useful information to integrate existDB in other CI platforms as well.
It is language independent but was tested only with Java and nodeJS. Both setups are provided.
You can test applications, that are served from the db or consume one of its APIs.
## tl;dr
CI? You should have it.
The complete setup to test anything against one or more versions of eXistDB on TravisCI with caching enabled
## Preface
I was recently working on node-exist. It is a node package, that consumes eXist's RPC API. In order to run the tests the database needed to be up.
Mocking of the database responses could have solved that problem, but…
Now you got 2 problems
- how to test against another version or multiple ones
- validating mock responses (of multiple versions)
A much better solution is for the tests to run on a continous integration platform. Here, every commit can be tested against different versions of the database in parallel. And, there is no need to have them runnning on the development machine. Plus, everyone with access can verify if a certain build is running and which database versions are supported by your application.
There are more arguments pro continuous integration, which you can easily find online.
TravisCI seemed to be a reasonable choice, not only because eXistDB itself runs its automated tests here.
## Which version to test against?
As travis automatically starts one build per entry in env your tests will be run against the 3.0.RC1 and the 2.2 release of eXist with
```yaml env: - EXISTDBVERSION=eXist-3.0.RC1 - EXISTDBVERSION=eXist-2.2 ```
Add or remove versions as you need them.
## before install
It proved to be handy to store the installation folder of the current DB version in an environment variable `EXISTDBFOLDER`. It will be used by following scripts and commands.
```yaml before install: - export EXISTDBFOLDER=$HOME/exist/${EXISTDBVERSION} ```
## installation and setup
To install the database we download the source from github, extract it and then call its build routine.
```sh export TARBALLURL=https://github.com/eXist-db/exist/archive/${EXISTDB_VERSION}.tar.gz
| mkdir -p ${EXISTDBFOLDER} | |
| curl -L $TARBALL_URL | tar xz -C ${EXISTDBFOLDER} --strip-components=1 |
| cd ${EXISTDBFOLDER} | |
| ./build.sh | |
| ``` |
All of the above can be nicely packed into a setup script.
`yaml install: - ci/setup-db.sh `
Note: `EXISTDBVERSION` is the environment variable we defined at the very beginning. This can be a tag or branchname or even a commit hash.
## Can't we start already?
Yes, but in order to do that we have to start eXist in the background and wait for it to listen to requests. Last, ensure that by doing a very simple one.
```sh cd ${EXISTDBFOLDER} nohup bin/startup.sh & sleep 30 curl http://127.0.0.1:8080/exist ```
Yes, you guessed it. There is a database start-up script.
`yaml before_script: - ci/start-db.sh `
Note: Without changing to its installation folder a bunch of exceptions will end up in nohup.out complaining about log files that cannot be found and opened.
## That's it
Run your tests!
```yaml script: - do test / ```
Now it is time to clean up the closet:
```yaml afterscript: - cd ${EXISTDB_FOLDER} - bin/shutdown.sh ```
## Tweaks
### Caching
Downloading and building exist from source can take up to 3 minutes. So, you may want to speed up your tests by caching the built database. The archived cache has still to be loaded from s3 but you will gain an extra minute or so - YMMV.
#### Is this version of existDB already cached?
In our case that boils down to: does the folder exist?
```sh if [ -d "$EXISTDBFOLDER" ]; then echo "Using cached eXist DB instance: ${EXISTDBVERSION}." exit 0 fi ```
#### Teardown this Database
Remove any data that is or might be left behind by your tests and remove logs, too. To make sure that you will always get the latest version for branches or refs like HEAD, anything but releases should be excluded from caching.
```sh if [[ "${EXISTDBVERSION}" == eXist ]]; then echo "reset data and logfiles for ${EXISTDBVERSION}" cd ${EXISTDBFOLDER} ./build.sh clean-default-data-dir rm webapp/WEB-INF/logs/.log exit 0 # fi
echo "exclude ${EXISTDBVERSION} from cache" rm -rf ${EXISTDBFOLDER} ```
Put together:
```yaml before_cache: - ci/teardown-db.sh cache: directories: - $HOME/exist ```
## What if Java is not you first language?
If you are not testing a java-application, as I was, you need to install java 1.8 into to the testing container with:
`yaml addons: apt: packages: - oracle-java8-installer `
and make this version the default by adding
`yaml - export JAVA_HOME=/usr/lib/jvm/java-8-oracle `
to the before_install step.
## The Gist of the Story
(Integration) testing is necessary, so if you're application depends on eXistDB then you should definitely have a look at this project.
### contents
- .travis.yml (Java setup)
- node.travis.yml (nodeJS setup)
- ci/* (helper scripts described above)
- utility/* (ant, Java setup utility functions)
- project/* (sample project)