Re: [xml-dev] XQuery 2019 IMap

[Hans-Juergen Rennau <hrennau@yahoo.de>]

Liam, I thank you for your thoughtful comments, and for not being upset by my polemics.

In the beginning was the Word, and I believe the root cause of the problem is not committees and politics, but language. This is revealed by the difficulty to answer the crucial point you made:

"

But i think really what you have to show is not only excitement about
technical possibilities but how your proposal would answer business
objectives:
* do something you couldn't do before;
* do something you are already doing, but more cheaply
* do something you are already doing, but faster
* do something you are already doing, but more reliably
and so on. This is the kind of story that attracts funding and

interest.

"

XQuery has the potential to change our thinking about structured information, but to communicate this potential is difficult, due to the chicken egg problem. The appropriate language is not familiar to conventional thinking. Key terms are either unknown (e.g. navigation axis) or, even more detrimental, they are known but have little weight - for instance, "navigation" and "structural relationship". So pointing to a totally new model of navigation amounts to multiplying almost-zero with a large number, obtaining almost-zero. Saying that an unprecedented way of expressing structural relationships creates a new kind of building blocks for thinking about problems - is without effect.

As known terms are reduced to almost-zero, and having anyway lost all hope of being understood, I resort to new language made up ad hoc. I say:

"XQuery supports multi-source, forking, discontinuous, conditional, generative navigation."

LEMON CURRY? Exactly - multi-source, forking, discontinuous, conditional, generative navigation!

* Navigation? The mapping of information items mentally arranged in a spatial structure to other items within that structure.

* Multi-source? Given a navigation expression PATH, you can apply it to multiple input: $nodes/PATH

* Forking? Within the navigation process, any item reached at an intermediary stage can be mapped to any number of next items.

* Discontinuous? My word for any step which is not parent->child or parent->attribute, as conventional thinking does not imagine other steps.

* Conditional? Any step can be attached to conditions, aka predicates.

* Generative? The final step can map the navigation result to any kind of representation, e.g. CSV records.

And it should not be forgotten that all this has nothing whatsoever to do with XML. Let me give a concrete example.

* Given: a set of file system directories, containing JSON files. Deep down in their bowels, these contain a 'flights':{...} object containing 'flight':{...}, objects describing a flight in terms of departureAirport, arrivalAirport, departureTime, arrivalTime.

* Desired: a CSV file with corresponding columns, reporting all flights after a user-specified time and involving a user-specified airport.

* XQuery solution: 8 lines of code, four of which are spent on collecting user input (see below). 

To your questions:

* do something you couldn't do before; NO
* do something you are already doing, but more cheaply YES
* do something you are already doing, but faster YES

* do something you are already doing, but more reliably YES

(And I add an own question & answer:

* is the solution inspired by a new kind of thinking about structural relationships; YES)

To the new language of description:

* Navigation - obviously

* Multi-source - using as input all JSON file found in or at any level under a set of directories ($docs//...)

* Forking - navigating from 'flights':{} objects to all 'flight':{} child objects

* Discontinuous - moving along the descendant-or-self axis (//)

* Conditional - filtering by predicates

* Generative - using as final navigation step a call of fn:string-join, creating a CSV record

With kind regards

Hans-Juergen

PS: Eight lines of XQuery code transforming a set of file system directories filled with JSON files into a CSV file reporting flights from or to a given airport after a given time. Directories, file name pattern, airport and time are input variables. Using extension functions offered by BaseX (file:list and json:parse) which SHOULD be defined by XQuery 3.2.

=================================================================

declare variable $dirs external := '/projects/th/liam /projects/th/liam /projects/th/liam /projects/th/liam2';
declare variable $fname external := '*flights*.json';
declare variable $airport external := 'JFK';

declare variable $time external := '08:00';

declare variable $docs := $dirs ! normalize-space(.) ! tokenize(., ' ') !
                          file:list(., true(), $fname) ! unparsed-text(.) ! json:parse(.);
$docs//flights/_[departureTime > $time][$airport = (departureAirport, arrivalAirport)]
/string-join((departureAirport, arrivalAirport, departureTime, arrivalTime), ',')

=================================================================

Example output:

  FRA,JFK,20:00:00,22:00:00
  JFK,FRA,08:00:00,10:00:00

Am Samstag, 5. Januar 2019, 23:23:30 MEZ hat Liam R. E. Quin <liam@fromoldbooks.org> Folgendes geschrieben:

On Sat, 2019-01-05 at 10:23 +0000, Hans-Juergen Rennau wrote:
> Hello, hello, has anybody the postal address of Sir Tim Berners-Lee?
> I want to write him a letter, expressing my disappointment. I shall
> not mince words, trusting that truly great minds are not touchy. I
> want to ask - how can the closing of the XQuery Working Group be
> explained, if not by a headstrong refusal to think deeply?

It can be explained because the Working Group was down to very few
active participants. The W3C work is funded by Members joining W3C and
sending people to the Working Groups. So if Members don't go, the work
ends. The decision to close the Working Group wasn't Tim's, as it
happens.

> "XQuery" is a double lie - it is not about
> XML, and it is not a query language. It is about mapping information.

Although i agree with you about mapping information, the word lie is
perhaps a little loaded.

At one point i tried to push for a different name; perhaps unexcitingly
i think i suggested fast forest, and for people to talk about forest
stores rather than XML databases. But there was considerable push-back
from the Working Group.

[...]

> True, the generalization from XML to Information is still rudimentary

In general i think of XML as being an information representation
syntax; as RDF as being a knowledge representation model (in the sense
of 1970s and 1980s expert systems); of JSON as being an intechange
syntax for data and (less happily) configurations. But these are only
approximate, not clear boundaries.

An advantage of a syntax-based representation is that there's no single
fixed way to parse XML: you can build any number of data structures.
It's perfectly acceptable to read an SVG image and construct a colour
pallette, not the actual image.

> - we need standardized parsing of non-XML resources into XDM node
> trees.

Do we? Well, we have some ways to get from JSON to XDM in the XPath 3.1
Functions and Operators book.

> True, the sister paradigm of information structure, graph, is still
> not supported natively - it should be integrated into the XDM (adding
> RDF triples and RDF datasets) as well as the expression language
> (adding SPARQL expressions).

Once you go here i think you have indeed left behind the X.

> But these shortcomings just point to the need of an XQuery Working
> Group - for a continuation of the journey from XQuery to IMap..

(IMAP itself as a name is already in use for email, of course; i'm not
sure whether that matters)

Possible ways forward include a W3C community group, if you want to be
able to reawaken the XQuery Working Group (and presumably the XSLT
Working Group); an external-to-W3C organization (Oasis? or something
ad-hoc?).

But i think really what you have to show is not only excitement about
technical possibilities but how your proposal would answer business
objectives:
* do something you couldn't do before;
* do something you are already doing, but more cheaply
* do something you are already doing, but faster
* do something you are already doing, but more reliably
and so on. This is the kind of story that attracts funding and
interest.

Liam


--
Liam Quin - http://delightfulcomputing.com/
web slave for https://www.fromoldbooks.org/
with fabulous vintage art and fascinating texts to read.