XQuery in eXist-db

Supported Optional Features

In addition to the standard features, eXist-db provides extended support for modules .

It implements the full axis feature, providing support for the following optional axes: ancestor::, ancestor-or-self::, following::, following-sibling::, preceding:: and preceding-sibling::. The only optional axis not supported is the namespace axis.

Unsupported features

eXist-db supports strong typing whenever the expected type of an expression, function argument, or function return value is explicitly specified or can be known otherwise. However, the following schema- and data type-related features are not supported:

• schema import and schema validation features (defined as optional in the XQuery specification). Instead, eXist-db provides a validation module containing extension functions to perform XML validation.

• When storing XML documents, eXist-db does not store type information along with the nodes. It therefore doesn't know the typed value of a node and has to assume xs:untypedAtomic (as defined by the XQuery specification).

• eXist-db does not support specifying a data type in an element or attribute test. The node test element(test-node) is supported, but the type portion of the test element(test-node, xs:integer) is ignored.

eXist-db supports standard XQuery FLWOR clause constructs except for the following:

• "tumbling window" and "sliding window" clauses

• "count" clause

eXist-db supports all standard XQuery functions except for the following known issues:

• map:merge should default to "use-first" but eXist-db defaults to "use-last." The developers plan to adopt the standard option in the next major version of eXist.

eXist-db supports all standard serialization parameters, but some bugs remain in the implementation, as reported by XQuery Test Suite (XQTS). For the internal tests that pass, see the eXist-db serialization tests.

Using Relative URIs

If the location specified in an import statement is a relative URI, the query engine tries to load the module relatively to the current module load path. The module load path is determined as follows:

1. If the main XQuery was loaded from a database collection, the module load path is the URI of that collection. For example if you access an XQuery via the REST server:

   http://localhost:8080/exist/servlet/db/modules/test.xq

   Now all relative module paths will be resolved relative to the /db/modules collection.

2. If the main XQuery was retrieved from the file system, the module load path points to that directory. This applies to queries executed through the XQueryServlet, XQueryGenerator or the Java admin client.

Preloaded Modules

Preloaded modules do not need to be explicitly imported or declared in the prolog of queries. The <builtin-modules> element in conf.xml lists the namespaces and the corresponding Java class that implements all modules to be preloaded:

<xquery enable-java-binding="no">
  <builtin-modules>
    <module uri="http://exist-db.org/xquery/util" class="org.exist.xquery.functions.util.UtilModule"/>
    <module uri="http://exist-db.org/xquery/transform" class="org.exist.xquery.functions.transform.TransformModule"/>
  </builtin-modules>
</xquery>

EXPath Packages

The eXist-db package repository makes it easy to manage and deploy external packages (.xar archives) containing Java or XQuery libraries and even full eXist-db applications. See the Package Repository documentation.

Creating Modules in XQuery

You can declare an XQuery file as a module and import it using the standard import module directive. The XQuery engine imports each module only once during compilation. The compiled module is made available through the static XQuery context.

Creating Modules in Java

To register Java-based XQuery modules, eXist-db requires a namespace URI to identify it and the list of functions it supplies. For this, you need to pass a driver class to the XQuery engine. This class must implement the interface org.exist.xpath.InternalModule.

The class org.exist.xpath.AbstractInternalModule already provides an implementation skeleton. The class constructor expects an array of function definitions for all functions to registered. A function definition (class FunctionDef) has two properties: the static signature of the function (as an instance of FunctionSignature) and the Java Class that implements the function.

A function is a class extending org.exist.xquery.Function or org.exist.xquery.BasicFunction. Functions without special requirements (like overloading) should subclass BasicFunction. To illustrate this a simple function definition:

public class EchoFunction extends BasicFunction {

public final static FunctionSignature signature =
new FunctionSignature(
	new QName("echo", ExampleModule.NAMESPACE_URI, ExampleModule.PREFIX),
	"A useless example function. It just echoes the input parameters.",
	new SequenceType[] { 
	    new FunctionParameterSequenceType("text", Type.STRING, Cardinality.ZERO_OR_MORE, "The text to echo")
    },
	new FunctionReturnSequenceType(Type.STRING, Cardinality.ZERO_OR_MORE, "the echoed text"));

public EchoFunction(XQueryContext context) {
	super(context, signature);
}

public Sequence eval(Sequence[] args, Sequence contextSequence)
throws XPathException {
	// is argument the empty sequence?
	if (args[0].getLength() == 0)
		return Sequence.EMPTY_SEQUENCE;
	// iterate through the argument sequence and echo each item
	ValueSequence result = new ValueSequence();
	for (SequenceIterator i = args[0].iterate(); i.hasNext();) {
		String str = i.nextItem().getStringValue();
		result.add(new StringValue("echo: " + str));
	}
	return result;
}
}

• Note that every function class has to provide a function signature. This defines the QName by which the function is identified, a documentation string, the sequence types of all arguments and the sequence type of the returned value.

  In the example above, we accept a single argument named "text" of type xs:string and a cardinality of ZERO_OR_MORE with the description "The text to echo". In other words, we accept any sequence of strings containing zero or more items. The return value is of type xs:string and a cardinality of ZERO_OR_MORE with the description "the echoed text".

  A parameter description should be normal sentence starting with a capital letter. The return value description is always prefixed with "Returns ", so make sure the text matches.

• The subclass overwrites the eval method, which has two arguments: the first contains the values of all arguments passed to the function, the second passes the current context sequence (which might be null).

  Note that the argument values in the array args have already been checked to match the sequence types defined in the function signature. We therefore do not have to recheck the length of the array: if more or less than one argument were passed to the function, an exception would have been thrown before eval gets called.

  In XQuery, all values are passed as sequences. A sequence consists of one or more items, and every item is either an atomic value or a node. Furthermore, a single item is also a sequence. The function signature specifies that any sequence containing zero or more strings is acceptable for our method. We therefore have to check if the empty sequence has been passed. Here the function call returns immediately. Otherwise, we iterate through each item in the sequence, prefix echo: to its string value, and add it to the result sequence.

Now we want to add the function to a new module, so we have to provide a driver class. The driver class defines the namespace URI and default prefix for the module. Functions are registered by passing an array of FunctionDef to the constructor. The following is an example driver class definition:

public class ExampleModule extends AbstractInternalModule {

public final static String NAMESPACE_URI = 
    "http://exist-db.org/xquery/examples";
	
public final static String PREFIX = "example";
	
private final static FunctionDef[] functions = {
	new FunctionDef(EchoFunction.signature, EchoFunction.class)
};
	
public ExampleModule() {
	super(functions);
}

public String getNamespaceURI() {
	return NAMESPACE_URI;
}

public String getDefaultPrefix() {
	return PREFIX;
}

}

To use this in XQuery:

xquery version "1.0";

import module namespace example="http://exist-db.org/xquery/examples"
at "java:org.exist.examples.xquery.ExampleModule";

example:echo(("Hello", "World!"))

The query engine recognizes the java: prefix in the location URI and treats the remaining part org.exist.examples.xquery.ExampleModule as a fully qualified class name leading to the driver class of the module.

eXist-db specific options

These options are added in addition to the W3C options and can be set in the prolog, in fn:serialize#2 or file:sync#3 alongside the standard W3C ones. Their names have to be prefixed with "exist:" when used in fn:serialize#2 or file:sync#3 option maps. See also the fn:serialize section below.

jsonp : function-name (string)

Only relevant when the serialization method is set to json. Causes the JSON output to be wrapped in the named JSONP function.

output-doctype : ( "yes" | "no" )

Serializes the document's Document Type Declaration if one is present.

expand-xincludes : ( "yes" | "no" )

Must the serializer expand XInclude elements?

highlight-matches : [ both | elements | attributes | none ]

When querying text with the full text or n-gram extensions, the query engine tracks the exact position of all matches inside text content. The serializer can later use this information to mark those matches by wrapping them into an <exist:match> element.

Setting highlight-matches=both will enable this feature for every kind of indexable node.

process-xsl-pi : ( "yes" | "no" )

If a document is serialized and it has an XSL processing instruction, eXist-db can try to load the referenced stylesheet and apply it to the document.

add-exist-id : [ element | all | none ]

Shows the internal node ids of an element by adding an attribute exist:id="internal-node-id". Setting add-exist-id to element will only show the node id of the top-level element, all will show the ids of all elements.

json-ignore-whitespace-text-nodes : ( "yes" | "no" )

To discard white-space text nodes when serializing XML/XDM to JSON.

insert-final-newline : ( "yes" | "no" )

Add a final newline at the end of an XML or JSON document. This is especially useful when serializing to a git repository.

Legacy options:

Before serialization parameters were standardized in XQuery 3.0, eXist-db developed its own set of parameters. These are kept for backwards compatibility but should be considered deprecated as they can be replaced with W3C standard options.

method : [ xml | xhtml | json | text ]

Determines the serialization method. Should be one of xml, xhtml, json or text.

The xhtml method makes sure that XHTML elements with an empty content model are serialized in minimized form. For instance <img> will be output as <img/>.

Unlike the xml method, the xhtml setting uses only the short form for the elements declared empty in the xhtml DTD. For example, the <br> tag is always returned as <br/>.

The text method returns the contents of elements only. For instance, <A>Content</A> is returned as Content. Attribute values, processing instructions and comments are ignored.

media-type

The MIME content type of the output. It will be used to set the HTTP Content-Type header if the query is running in an HTTP context.

encoding

Specifies the character encoding to be used for outputting the instance of the data model

doctype-public and doctype-system

a doctype declaration will be output if doctype-public and/or doctype-system are set. The corresponding identifier is taken from the value of the parameter.

indent= yes | no

Indent the document to make it easier to read. Indenting adds whitespace characters to element nodes, restricted by the rules given in the XQuery serialization specification.

omit-xml-declaration= yes | no

Output an XML declaration if the parameter is set to no

Declaring eXist-db specific serialization parameters in prolog

Before serialization parameters were standardized in XQuery 3.0, eXist-db developed its own set of parameters, as described in below. These legacy parameters remain supported for backwards compatibility. For example, the eXist-db legacy approach to the JSON serialization declaration above is:

declare option exist:serialize "method=json media-type=application/json";

Most of the exist-db specific serialization parameters have an equivalent in the standard serialization parameters, but others are truly specific to eXist-db.

To declare them in the prolog of a XQuery module use the declare option statement. The syntax differs from the W3C that all options are declared in a single statement. parameter=value pairs are separated by whitespace in a single string. For instance:

declare option exist:serialize "method=xhtml media-type=application/xhtml+html";

The option's QName must be exist:serialize. The exist prefix is bound to the namespace http://exist.sourceforge.net/NS/exist. This is declared by default and need not be specified explicitly.

Options are specified within the option's string literal, separated by whitespace.

For example, to disable XInclude expansion and indent the output, use the following syntax:

declare option exist:serialize "expand-xincludes=no indent=yes";

Using fn:serialize()

Another Serialization method that can be used is in-built function fn:serialize().

You can apply eXist-db specific serialization parameters as well as W3C serialization parameters via a map.

fn:serialize($node, map { "exist:expand-xincludes": false() })

For W3C serialization parameters check:W3C

eXist-db specific options:

Check Serialize Tests to see the options in action

jsonp: function-name as xs:string

fn:serialize($node, map {"method":"json", "media-type":"application/json", "exist:jsonp":"functionName"})

output: 'functionName({})'

expand-xincludes: xs:boolean

fn:serialize($doc, map { "exist:expand-xincludes": true() })

expand XInclude elements

highlight-matches: "both" | "elements" | "attributes" | "none"

fn:serialize($hit, map { "exist:highlight-matches": "both" })

A complete example Serialize Test with ft:query()

process-xsl-pi: xs:boolean

fn:serialize($doc, map { "exist:process-xsl-pi": true() })

add-exist-id: "element" | "all" | "none"

fn:serialize($doc, map { "exist:add-exist-id": "all" })

insert-final-newline: xs:boolean

fn:serialize($doc, map { "exist:add-exist-id": "all" })

Query Blocking

To prevent the server from being blocked by a badly formulated query, eXist-db watches all query threads. A blocking query can be killed if it takes longer than a specified amount of time or consumes too many memory resources on the server. There are two options to control this behaviour:

declare option exist:timeout "time-in-ms";

Specifies the maximum amount of query processing time (in milliseconds) before it is cancelled by the XQuery engine.

declare option exist:output-size-limit "size-hint";

Defines a limit for the maximum size of a document fragment created within an XQuery. The limit is just an estimation, specified in terms of the accumulated number of nodes contained in all generated fragments. This can be used to prevent users from consuming too much memory if they are allowed to pass in their own XQueries.

Overriding the default XQuery Context

eXist-db provides the following options to override the default XQuery context:

declare option exist:implicit-timezone "duration";

Specifies the implicit timezone for the XQuery context.

declare option exist:current-dateTime "dateTime";

Specifies the current dateTime for the XQuery context.

declare option exist:optimize "enable=yes|no";

Temporarily disables the query rewriting optimizer for the current query. Use for testing/debugging.

HTTP Client

eXist-db implements the HTTP Client Module, an EXPath specification for interacting with HTTP servers.

Packaging System

eXist-db implements the Packaging System, an EXPath community specification for library and application packages. See the Package Repository documentation.

RESTXQ

eXist-db implements RESTXQ, an EXQuery community effort to develop a clean approach for the deployment of RESTful services based on XQuery code annotations. This is in addition to eXist-db's native function libraries for performing HTTP requests and responses.

ZIP

eXist-db implements the ZIP Module, an EXPath community specification and module for performing compression. This is in addition to eXist-db's native function libraries for performing compression.