Parallel execution: functions, XPath, XQuery,

[Ken North <ken_north@compuserve.com>]

A review of the current working documents for XQuery and XPath 2.0
requirements raises some questions about building a search engine that can
be deployed over a parallel processing architecture.

Because SQL query optimizers can "parallelize" operations, SQL engines can
exploit parallel architectures and use multiple processors for
query evaluation and execution. Clearly we want XML processing to be able to
exploit massively parallel processing, clusters, and other parallel
architectures, but there are issues:

1. SQL is set-oriented so it's possible to partition data across
(processing) nodes, partition queries, and combine results. An SQL query
doesn't define a navigation path for getting to data. The query just
specifies what's wanted and a query optimizer determines the most efficient
means of execution.

So issue 1 is how to partition document collections for optimal parallel
execution with XPath and XQuery. I've not found any published research on
parallel evaluation or parallel execution with XPath. Anyone have a URL?

2. The XPath 2.0 requirements document identifies the need for additional
string and aggregate functions, but doesn't define a requirement for being
able to invoke user-defined functions (UDFs). XQuery supports
XPath functions but it also supports UDFs. A future version will support
UDFs implemented in programming languages.

So issue 2 is the future direction of XPath, particularly functions and
extensibility.

3. If you are using an SQL products such as Oracle 8i and IBM DB2, you can
add UDFs for processing XML data by implementing them in Java, C/C++ and
deploying them as a database plug-in. For example, by implementing FOO() in
Java, I can use SELECT FOO() in SQL queries. It's possible to use FOO() in
parallel queries. DB2, for example, provides an ALLOW PARALLEL option when I
register my function classes with the database. Therefore, SQL queries can
still be "parallelized" if they include my Java, C/C++ UDFs.

So issue 3 is how to express to XPath or XQuery details such as whether to
use a single instance of the Java class per query or allow parallel use of
an UDF.