1. Atomic Wiki
  2. Blogs
  3. eXist Developer's Blog
  4. eXist-db 4.3.0

eXist-db 4.3.0

by wolf at 2018/07/06 at 21:07:28 z
releasearticle

v4.3.0 - July 06, 2018

eXist-db v4.3.0 has just been released. This is a feature release, which contains several new features and bug fixes.

We recommend that all users of eXist 4.x.x should upgrade to eXist 4.3.0.

Features

  • Upgraded to Apache Xerces 2.12.0. eXist-db now includes XML Schema 1.1 support.
  • Improved the Remote XML:DB API "networking error" messages to be more descriptive.
  • Test errors are now summarised in XQSuite XML output.

Bug fixes

  • Fixed an issue where in-memory documents used in XQuery Update insert expressions caused an NPE (NullPointerException).
  • Fixed an issue where XQuery Error Codes were not correctly reported under some circumstances.
  • Fixed an issue with restoring binary files of zero bytes via the XML-RPC and XML:DB Remote API.
  • Fixed a signing issue with running eXist-db from the installer under Java 10.
  • Fixed an issue when constructing in-memory documents with namespaces via SAX parsers.
  • Fixed an issue with namespaces when parsing external documents to in-memory documents.
  • Fixed an issue with namespaces when merging in-memory documents.
  • Fixed a NPE when calling map:get for a non-existent key/value map pair.
  • Fixed a NPE in fn:data.
  • Fixes a NPE when converting empty sequences.

Backwards Compatibility

  • eXist-db v4.3.0 is backwards binary-compatible as far as eXist-db v3.0, but not with earlier versions. Users who are upgrading should always consult the [Upgrading Guide](http://exist-db.org/exist/apps/doc/upgrading.xml) in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db
  • 6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.3.0, you
  • MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.3.0, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.3.0 and use Dashboard.
    2. If you install eXist-db 4.3.0 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.

Downloading This Version

  • eXist-db v4.3.0 is available for download from GitHub.
  • Docker images are available from EvolvedBinary's DockerHub.
  • Maven artifacts are available from our mvn-repo.
  • Mac users of the Homebrew package repository may acquire eXist 4.3.0 directly from there.
  1. Atomic Wiki
  2. Blogs
  3. eXist Developer's Blog
  4. eXist-db 5.0.0 RC 1

eXist-db 5.0.0 RC 1

by wolf at 2018/06/14 at 11:07:28 z
releasearticle

# Release Notes

## v5.0.0-RC1 - June 9, 2018

We are very excited to announce the first Release Candidate (RC) for eXist-db 5.0.0.

NOTE: A Release Candidate is not recommended for production use. We have tested the release extensively, and we now invite all users to report both their positive and negative experiences with it. As always please make sure you have frequent and correct backups of your database.

The core improvements to eXist-db's locking, concurrency, and transactions in 5.0.0-RC1 represent a huge amount of work contributed by Evolved Binary. For in-depth information on these changes, see Evolved Binary's technical reports: Locking and Cache Improvements for eXist-db and Asymmetrical Locking for eXist-db.

In eXist-db 5.0.0-RC1, all locking interactions have been redesigned and reimplemented from the ground up; a number of deadlock paths have been removed; and many operations have been redesigned to be thread-safe. The result is that eXist-db should exhibit better responsiveness under concurrent user operations. For example, because collection locks are now multi-read capable, re-indexing the database no longer blocks access to the collection being re-indexed. Users should notice improved responsiveness during such operations.

5.0.0-RC1 also contains some security fixes and bug fixes that necessitated API changes and thus were not suitable for the existing 4.x.x line.

### Highlights Include

  • Hierarchical locking strategy for Collection locks.
  • Centralised Lock Manager, which feeds a Lock Table for reporting to the user on lock leases and pending lock acquisitions (with JMX support).
  • All bespoke locks have been removed and/or replaced with standard Java library locks.
  • Managed Lock wrappers for ARM (Automatic Resource Management), to ensure locks are never left dangling.
  • Most operations now exploit deadlock avoidance strategies.
  • Asymmetrical locking between Collection and Document locks; Collection locks now have shorter leases.
  • Collections now exploit read/write locking, rather than Mutex locking; multiple reads can happen in parallel.
  • Complete rewrite of Collection Caching using Ben Mane's Caffeine library.
  • Reduced Transaction overhead, by reusing existing transactions.
  • Updated Saxon for XSLT to 9.8.0-12.
  • XQuery function signatures are more closely aligned to the W3C XQuery specification.
  • util:eval-with-context now offers a timeout argument.
  • Copy APIs now offer a preserve flag, which behaves similarly to GNU cp's --preserve flag.
  • Implemented simplified xmldb:copy-resource and xmldb:copy-collection functions.

### Security fixes

  • Fixed some incorrectly relaxed owner and mode permissions on copied Collections and Documents. #1888.
  • Fixed some incorrect ownership and mode problems with chown and chmod operations. #1893.

### Bug fixes

  • The last used User and Group id's are no longer exported during a database Backup, nor are they read back from a Backup when Restoring. As such the format of the file /db/system/security/config.xml has changed. #1858.
  • XML-RPC and XML:DB Remote APIs now use GZip encoding for streams and not Zip. This introduces a non-backwards compatible change to the XML-RPC API which also affects the Java Admin Client. #1903.
  • Numerous erroneous locking states (see Evolved Binary's Technical Reports).

### Backwards Compatibility

  1. The format of /db/system/security/config.xml has changed. You can restore Backups from previous versions of eXist-db. However, you cannot restore a Backup from eXist-db 5.0.0-RC1 to a previous version of eXist-db.
  2. The XML-RPC API is not 100% backwards compatible with prior versions of eXist-db, particularly with regards to uploading/downloading documents. You should always use the correct version of the Java Admin Client with eXist-db.
  3. Versions of the Dashboard App prior to version 1.1.0 are not compatible with eXist-db 5.0.0-RC1. If you wish to migrate from an older version of eXist-db, you must first update both the Shared Resources package and the Dashboard package on your existing eXist-db version to at least versions 0.6.0 and 1.1.0 respectively. If you are starting with a new clean install of eXist-db 5.0.0-RC1 you have nothing to worry about.

eXist-db v5.0.0-RC1 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.

### Downloading this Version * eXist-db v5.0.0-RC1 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 5.0.0-RC1 directly from there.
  1. Atomic Wiki
  2. Blogs
  3. eXist Developer's Blog
  4. eXist-db 4.2.1

eXist-db 4.2.1

by wolf at 2018/06/14 at 11:07:28 z
releasearticle

## v4.2.1 - June 14, 2018

eXist-db v4.2.1 has just been released. This is a hotfix release, which contains bug fixes for several important issues discovered since eXist-db v4.2.1.

We recommend that all users of eXist 4.2.0 should upgrade to eXist 4.2.1.

### Bug fixes * Fixed an NPE with XQuery Update insert preceding with attributes. Closes #1761 and #1939.
  • Only attributes which are considered boolean in HTML5 have their values omitted. Closes #777. Thanks to Bert Willems from FontoXML for this fix.
  • Fixed an issue with the XQuery function inspect:module-functions, which was previously throwing a NPE. Closes #1927.
  • Fixed an issue with restoring Compressed (Zip) database backups. Closes Closes #1913.

### Backwards Compatibility

  • eXist-db v4.2.1 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db 3.6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.2.1, you MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.2.1, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.2.1 and use Dashboard.
    2. If you install eXist-db 4.2.1 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.
### Downloading This Version * eXist-db v4.2.1 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 4.2.1 directly from there.
  1. Atomic Wiki
  2. Blogs
  3. eXist Developer's Blog
  4. eXist-db 4.2.0

eXist-db 4.2.0

by wolf at 2018/06/06 at 08:41:28 z
releasearticle

# Release Notes

## v4.2.0 - June 6, 2018

eXist-db v4.2.0 has just been released. This is a minor release, which contains an important security fix, as well as several new features and bug fixes.

### Security Fixes Added avoidance for Zip Slip / Zip Exit attacks. NOTE: Developers should take care when writing their own $entry-data functions for compression:unzip or compression:untar if they are writing the file entries to persistent storage using the name of the entry from the archive; Consideration should be given to employing further exit attack protections of their own.
### Features XQuery compression:unzip and compression:untar functions have received some lower-arity signatures which should simplify use. Also some helper functions have been added: compression:no-filter, `compression:db-store-, and compression:fs-store-`. The XQuery util:eval-with-context function now allows a timeout to be set. XQSuite gains the annotation %test:assumeInternetAccess for ensuring Internet Access is XML-RPC Server character set is now configurable, defaults to UTF-8. XQueryURLRewrite challenge is now configurable, defaults to always challenging for basic authentication. Building YAJSW no longer requires `JAVAHOME if java is available on the system PATH`. ClassLoaderSource now tracks source file for easy referencing. Display the specific JVM version details when building eXist-db. Updated third-party dependencies: Apache Commons Configuration 1.17 Java 8 Functional Utilities 1.16 FasterXML Jackson 2.9.5 Jargo 0.4.14 XMLUnit 2.6.0 Eclipse Jetty 9.4.10.v20180503 * Eclipse AspectJ 1.9.1
## Performance Improvements Minor performance improvements when streaming binary files from the database. Replaced many uses of Reflection with use of Constructor lambda functions and Method Handles. Broker construction functions are now cached. In the XML:DB Remote API small transfers will not be compressed, which saves both time and space (due to compression overhead). * XML:DB Remote API no longer allocates full-chunks in memory if they are not required.
### Bug Fixes Fixed a memory leak which manifests when making heavy use of higher-order-functions in XQuery. Fixed a problem with serialization where empty namespace declarations could be lost. This applies to the XML:DB, XML-RPC and WebDAV interfaces. SAX Serialization now handles lexical CDATA sections and DTD's. Fixed a potential deadlock when validating XML documents against various types of Schema. EXPath HTTP Client now correctly adheres to the character set of the server's response. Reduced severity of log messages in the Temporary File Manager. Memory Mapping of temporary files is now handled correctly on Microsoft Windows. One eXist-db instance can no longer accidentally delete the temporary files of another instance (which may be in use). Fixed concurrent use issues with the Index Manager. Index configuration changes now cause the Index Controller to be reloaded. fn:format-date previously calculated the name of the day of the week incorrectly (off by one e.g. Monday -> Tuesday). file:mkdirs no longer incorrectly raises an error if the directory already exists. Fixed a NullPointerException when uploading large documents via the XML-RPC and XML:DB Remote APIs. Handling of Zip entry filenames are now compatible with Java 10. Fixed the behaviour of the XMLUnit functions. XQSuite now correctly displays results when an error is expected. XQSuite now shows the correct messages for %test:assertTrue and %test:assertFalse. XSuite test resources are now correctly loaded from the classpath.

### Backwards Compatibility

  • eXist-db v4.2.0 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db 3.6.1 (or earlier) is incompatible with eXist-db 4.0.0 and newer. If you plan to migrate your entire database to eXist-db 4.2.0, you MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.2.0, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.2.0 and use Dashboard.
    2. If you install eXist-db 4.2.0 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.
### Downloading This Version * eXist-db v4.2.0 is available for download from GitHub. * Docker images for are available from EvolvedBinary's DockerHub. * Maven artifacts are available from our mvn-repo. * Mac users of the Homebrew package repository may acquire eXist 4.2.0 directly from there.
  1. Atomic Wiki
  2. Blogs
  3. eXist Developer's Blog
  4. eXist-db 4.1.0

eXist-db 4.1.0

by wolf at 2018/04/16 at 17:41:28 z
releasearticle

Release Notes

v4.1.0 - April 16, 2018

eXist-db v4.1.0 has just been released. This is a minor release, which contains important bug fixes. Updating to this release is recommended to users experiencing any of the issues below:

Important Bug Fixes

  • A long standing bug caused 0-byte XML documents to appear in database collections. The bug occurred randomly. The document showed up in the collection but was empty when opened.
  • Wrong nodes being returned as the result of an XQuery predicate using a range index. Serializing the returned sequence would show seemingly random nodes from the same document.
  • Several issues and regressions concerning handling binary resources have been fixed.

Features

  • XSuite - Proper integration between JUnit, and XQSuite and XML XQuery tests.
  • Add more Travis CI JDKs (including Java 9).
  • Support more locales for month and day names in format-date.
  • Add support for charset encoding from http response.
  • Updates:
    • Jetty 9.4.9.v20180320
    • YAJSW 12.12
    • AspectJ 1.9.0.RC4
    • cglib 3.2.6
    • j8fu 1.13
    • Jackson 2.9.4
    • jctools 2.1.2
    • log4j 2.11.0
    • Commons Compress 1.16.1
    • Apache HTTP Components 4.5.5

Performance Improvements

  • Switch to non-concurrent ByteArrayOutputStream.

Bug Fixes

  • Fix regression with binary values from files and strings.
  • Fix parsing of QNames in XQuery: The XQuery parser ignored whitespace when parsing a qname, leading to a wrong interpretation of ':' in some contexts, e.g. map constructors.
  • Close input stream in fn:json-doc.
  • Enable previously disabled tests.
  • Fix wildcard lookup in ft:query.
  • Align xml:id handling with saxon and basex.
  • Closing inputstreams and improve ByteArrayOutputStream usage.

Apps

The bundled applications of eXide, and Monex have all been updated to the latest versions.

Backwards Compatibility

  • eXist-db v4.1.0 is backwards binary-compatible as far as v3.0, but not with earlier versions. Users who are upgrading should always consult the Upgrading Guide in the documentation.
  • NOTE: The version of Dashboard that was installed with eXist-db 3.6.1 (or earlier) is incompatible with eXist-db 4.0.0 and later. If you plan to migrate your entire database to eXist-db 4.1.0, you MUST take one of the following steps to allow Dashboard to work after the upgrade:
    1. If you have not yet installed eXist-db 4.1.0, open Dashboard > Package Manager, and upgrade Dashboard to 0.4.10, or run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may safely install eXist-db 4.0.0 and use Dashboard.
    2. If you install eXist-db 4.1.0 before upgrading Dashboard, run this script in eXide or the Java Admin Client: repo:install-and-deploy("http://exist-db.org/apps/dashboard", "http://demo.exist-db.org/exist/apps/public-repo/modules/find.xql"). Then you may use Dashboard.

Downloading This Version

  • eXist-db v4.1.0 is available for download from Bintray.
  • Docker images for are available from EvolvedBinary's DockerHub.
  • Maven artifacts are available from our mvn-repo.
  • Mac users of the Homebrew package repository may acquire eXist 4.1.0 directly from there.