eXist-db 5.0.0 RC 2

## v5.0.0-RC2 - July 25, 2018

The second Release Candidate (RC) for eXist-db 5.0.0 has been released.

eXist-db 5.0.0-RC2 builds atop eXist-db 5.0.0-RC1 with a few new features, important bug fixes, and some small API changes.

  • NOTE: A Release Candidate is not recommended for production use. We have tested the release extensively, and we now invite all users to report both their positive and negative experiences with it. As always please make sure you have frequent and correct backups of your database.

### Features

  • Implemented basic support for "available collections" in the XQuery dynamic context.
  • Implemented basic support for "available documents" in the XQuery dynamic context.
  • Implemented basic support for "available text resources" in the XQuery dynamic context.
  • Implemented support for the collation used in the XQTS (XQuery Test Suite) 3.1.
  • Incorporated from 4.3.0:
    • Upgraded to Apache Xerces 2.12.0. eXist-db now includes XML Schema 1.1 support.
    • Improved the Remote XML:DB API "networking error" messages to be more descriptive.
    • Test errors are now summarised in XQSuite XML output.

### Bug fixes

  • Fixed a locking issue in the XML:DB Local API when storing XML documents.
  • Fixed an NPE which manifested when storing binary documents when preserve-on-copy was enabled.
  • Incorporated from 4.3.1:
    • Fixed a regression introduced in eXist-db 4.3.0, where the type conversion of non-empty XDM Sequences was incorrectly returning an empty sequence. This could lead to insufficient results being returned by an XQuery.
    • Fixed another problem with uploading zero length binary files via. the XML:DB Remote API.
    • Fixed a number of issues with the LDAP Security Realm:
      1. Ensure the authenticated LDAP context is reused.
      2. Always close and dispose of any LDAP context that we create.
      3. Fixed encoding of CN and DN names during LDAP searches.
    • Fixed an issue where the XQuery function file:serialize could use the incorrect character encoding depending on which platform eXist-db was being executed on.
    • Fixed a comparison problem with node equality in the DOM API exposed via. the XML:DB Local API.
    • Fixed a java.nio.file.Path issue when loading eXist-db's conf.xml file from inside a classpath Jar file.
    • Removed unnecessary error level logging of the XQuery fn:error function.
    • Fixed an issue with the error message for restoring permissions; Now the entire resource URI is shown.
    • Fixed an initialise/reset issue with Internal Module XQuery variables. This fixed a regression introduced in eXist-db 3.5.0.
    • Fixed a JavaEE dependency runtime issue with Java 11.
    • Downloading of XAR files during the eXist-db build process is now more descriptive.
  • Incorporated from 4.3.0:
    • Fixed an issue where in-memory documents used in XQuery Update insert expressions caused an NPE (NullPointerException).
    • Fixed an issue where XQuery Error Codes were not correctly reported under some circumstances.
    • Fixed an issue with restoring binary files of zero bytes via the XML-RPC and XML:DB Remote API.
    • Fixed a signing issue with running eXist-db from the installer under Java 10.
    • Fixed an issue when constructing in-memory documents with namespaces via SAX parsers.
    • Fixed an issue with namespaces when parsing external documents to in-memory documents.
    • Fixed an issue with namespaces when merging in-memory documents.
    • Fixed a NPE when calling map:get for a non-existent key/value map pair.
    • Fixed a NPE in fn:data.
    • Fixes a NPE when converting empty sequences.
  • Incorporated from 4.2.1:
    • Fixed an NPE with XQuery Update insert preceding with attributes. Closes #1761 and #1939.
    • Only attributes which are considered boolean in HTML5 have their values omitted. Closes #777. Thanks to Bert Willems from FontoXML for this fix.
    • Fixed an issue with the XQuery function inspect:module-functions, which was previously throwing a NPE. Closes #1927.
    • Fixed an issue with restoring Compressed (Zip) database backups. Closes #1913.
### API Changes * Users who have XQuery Modules written in Java which implement org.exist.xquery.InternalModule should no longer call InternalModule#declareVariable from their class constructor, instead they should place those calls in InternalModule#prepare(XQueryContext).
### Backwards Compatibility Apart from the small API changes described above, eXist-db 5.0.0-RC2 is backwards compatible with eXist-db 5.0.0-RC1. For details of backwards compatibility with versions of eXist-db prior to 5.0.0-RC1 please see the Backwards Compatibility details of 5.0.0-RC1.
### Downloading this Version * eXist-db v5.0.0-RC2 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 5.0.0-RC2 directly from there via. the exist-db-rc cask.

eXist-db 4.3.1

v4.3.1 - July 24, 2018

eXist-db v4.3.1 has just been released. This is a hot-fix release, which contains several important bug fixes.

We recommend that all users of eXist-db 4.x.x should upgrade to eXist-db 4.3.1.

Bug fixes

  • Fixed a regression introduced in eXist-db 4.3.0, where the type conversion of non-empty XDM Sequences was incorrectly returning an empty sequence. This could lead to insufficient results being returned by an XQuery.
  • Reverted Apache Xerces 2.12.0 to 2.11.0. This fixes a regression introduced in eXist-db 4.3.0, whereby Xerces 2.12.0 proved to be incompatible with betteForm. Apache Xerces 2.12.0 is available in eXist-db 5.0.0-SNAPSHOT, and will be available from eXist-db 5.0.0-RC2 onwards.
  • Fixed another problem with uploading zero length binary files via. the XML:DB Remote API.
  • Fixed a number of issues with the LDAP Security Realm:
    1. Ensure the authenticated LDAP context is reused.
    2. Always close and dispose of any LDAP context that we create.
    3. Fixed encoding of CN and DN names during LDAP searches.
  • Fixed an issue where the XQuery function file:serialize could use the incorrect character encoding depending on which platform eXist-db was being executed on.
  • Fixed a comparison problem with node equality in the DOM API exposed via. the XML:DB Local API.
  • Fixed a java.nio.file.Path issue when loading eXist-db's conf.xml file from inside a classpath Jar file.
  • Removed unnecessary error level logging of the XQuery fn:error function.
  • Fixed an issue with the error message for restoring permissions; Now the entire resource URI is shown.
  • Fixed an initialise/reset issue with Internal Module XQuery variables. This fixed a regression introduced in eXist-db 3.5.0.
  • Fixed a JavaEE dependency runtime issue with Java 11.
  • Downloading of XAR files during the eXist-db build process is now more descriptive.

Backwards Compatibility

  • eXist-db v4.3.1 is backwards binary-compatible as far as eXist-db v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db 3.6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.3.1, you MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.3.1, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.3.1 and use Dashboard.
    2. If you install eXist-db 4.3.1 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.

Downloading This Version

  • eXist-db v4.3.1 is available for download from GitHub.
  • Docker images are available from EvolvedBinary's DockerHub.
  • Maven artifacts are available from our mvn-repo.
  • Mac users of the Homebrew package repository may acquire eXist 4.3.1 directly from there.

eXist-db 4.3.0

v4.3.0 - July 06, 2018

eXist-db v4.3.0 has just been released. This is a feature release, which contains several new features and bug fixes.

We recommend that all users of eXist 4.x.x should upgrade to eXist 4.3.0.

Features

  • Upgraded to Apache Xerces 2.12.0. eXist-db now includes XML Schema 1.1 support.
  • Improved the Remote XML:DB API "networking error" messages to be more descriptive.
  • Test errors are now summarised in XQSuite XML output.

Bug fixes

  • Fixed an issue where in-memory documents used in XQuery Update insert expressions caused an NPE (NullPointerException).
  • Fixed an issue where XQuery Error Codes were not correctly reported under some circumstances.
  • Fixed an issue with restoring binary files of zero bytes via the XML-RPC and XML:DB Remote API.
  • Fixed a signing issue with running eXist-db from the installer under Java 10.
  • Fixed an issue when constructing in-memory documents with namespaces via SAX parsers.
  • Fixed an issue with namespaces when parsing external documents to in-memory documents.
  • Fixed an issue with namespaces when merging in-memory documents.
  • Fixed a NPE when calling map:get for a non-existent key/value map pair.
  • Fixed a NPE in fn:data.
  • Fixes a NPE when converting empty sequences.

Backwards Compatibility

  • eXist-db v4.3.0 is backwards binary-compatible as far as eXist-db v3.0, but not with earlier versions. Users who are upgrading should always consult the [Upgrading Guide](http://exist-db.org/exist/apps/doc/upgrading.xml) in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db
  • 6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.3.0, you
  • MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.3.0, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.3.0 and use Dashboard.
    2. If you install eXist-db 4.3.0 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.

Downloading This Version

  • eXist-db v4.3.0 is available for download from GitHub.
  • Docker images are available from EvolvedBinary's DockerHub.
  • Maven artifacts are available from our mvn-repo.
  • Mac users of the Homebrew package repository may acquire eXist 4.3.0 directly from there.

eXist-db 5.0.0 RC 1

# Release Notes

## v5.0.0-RC1 - June 9, 2018

We are very excited to announce the first Release Candidate (RC) for eXist-db 5.0.0.

NOTE: A Release Candidate is not recommended for production use. We have tested the release extensively, and we now invite all users to report both their positive and negative experiences with it. As always please make sure you have frequent and correct backups of your database.

The core improvements to eXist-db's locking, concurrency, and transactions in 5.0.0-RC1 represent a huge amount of work contributed by Evolved Binary. For in-depth information on these changes, see Evolved Binary's technical reports: Locking and Cache Improvements for eXist-db and Asymmetrical Locking for eXist-db.

In eXist-db 5.0.0-RC1, all locking interactions have been redesigned and reimplemented from the ground up; a number of deadlock paths have been removed; and many operations have been redesigned to be thread-safe. The result is that eXist-db should exhibit better responsiveness under concurrent user operations. For example, because collection locks are now multi-read capable, re-indexing the database no longer blocks access to the collection being re-indexed. Users should notice improved responsiveness during such operations.

5.0.0-RC1 also contains some security fixes and bug fixes that necessitated API changes and thus were not suitable for the existing 4.x.x line.

### Highlights Include

  • Hierarchical locking strategy for Collection locks.
  • Centralised Lock Manager, which feeds a Lock Table for reporting to the user on lock leases and pending lock acquisitions (with JMX support).
  • All bespoke locks have been removed and/or replaced with standard Java library locks.
  • Managed Lock wrappers for ARM (Automatic Resource Management), to ensure locks are never left dangling.
  • Most operations now exploit deadlock avoidance strategies.
  • Asymmetrical locking between Collection and Document locks; Collection locks now have shorter leases.
  • Collections now exploit read/write locking, rather than Mutex locking; multiple reads can happen in parallel.
  • Complete rewrite of Collection Caching using Ben Mane's Caffeine library.
  • Reduced Transaction overhead, by reusing existing transactions.
  • Updated Saxon for XSLT to 9.8.0-12.
  • XQuery function signatures are more closely aligned to the W3C XQuery specification.
  • util:eval-with-context now offers a timeout argument.
  • Copy APIs now offer a preserve flag, which behaves similarly to GNU cp's --preserve flag.
  • Implemented simplified xmldb:copy-resource and xmldb:copy-collection functions.

### Security fixes

  • Fixed some incorrectly relaxed owner and mode permissions on copied Collections and Documents. #1888.
  • Fixed some incorrect ownership and mode problems with chown and chmod operations. #1893.

### Bug fixes

  • The last used User and Group id's are no longer exported during a database Backup, nor are they read back from a Backup when Restoring. As such the format of the file /db/system/security/config.xml has changed. #1858.
  • XML-RPC and XML:DB Remote APIs now use GZip encoding for streams and not Zip. This introduces a non-backwards compatible change to the XML-RPC API which also affects the Java Admin Client. #1903.
  • Numerous erroneous locking states (see Evolved Binary's Technical Reports).

### Backwards Compatibility

  1. The format of /db/system/security/config.xml has changed. You can restore Backups from previous versions of eXist-db. However, you cannot restore a Backup from eXist-db 5.0.0-RC1 to a previous version of eXist-db.
  2. The XML-RPC API is not 100% backwards compatible with prior versions of eXist-db, particularly with regards to uploading/downloading documents. You should always use the correct version of the Java Admin Client with eXist-db.
  3. Versions of the Dashboard App prior to version 1.1.0 are not compatible with eXist-db 5.0.0-RC1. If you wish to migrate from an older version of eXist-db, you must first update both the Shared Resources package and the Dashboard package on your existing eXist-db version to at least versions 0.6.0 and 1.1.0 respectively. If you are starting with a new clean install of eXist-db 5.0.0-RC1 you have nothing to worry about.

eXist-db v5.0.0-RC1 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.

### Downloading this Version * eXist-db v5.0.0-RC1 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 5.0.0-RC1 directly from there.

eXist-db 4.2.1

## v4.2.1 - June 14, 2018

eXist-db v4.2.1 has just been released. This is a hotfix release, which contains bug fixes for several important issues discovered since eXist-db v4.2.1.

We recommend that all users of eXist 4.2.0 should upgrade to eXist 4.2.1.

### Bug fixes * Fixed an NPE with XQuery Update insert preceding with attributes. Closes #1761 and #1939.
  • Only attributes which are considered boolean in HTML5 have their values omitted. Closes #777. Thanks to Bert Willems from FontoXML for this fix.
  • Fixed an issue with the XQuery function inspect:module-functions, which was previously throwing a NPE. Closes #1927.
  • Fixed an issue with restoring Compressed (Zip) database backups. Closes Closes #1913.

### Backwards Compatibility

  • eXist-db v4.2.1 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db 3.6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.2.1, you MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.2.1, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.2.1 and use Dashboard.
    2. If you install eXist-db 4.2.1 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.
### Downloading This Version * eXist-db v4.2.1 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 4.2.1 directly from there.