RE: Data storage, data exchange, data manipulation

[Michael Brennan <Michael_Brennan@allegis.com>]

> From: Nicolas LEHUEN [mailto:nicolas.lehuen@ubicco.com]

<snip/>

> For example, you can process an XML document without 
> ID/IDREFs in a rather
> straightforward way using SAX. With ID/IDREF you have to 
> store a reference
> to every element with an ID of IDREF in order to resolve the 
> links in a
> second pass (e.g. if the IDREF precedes the ID). This is 
> quite expensive in
> memory and CPU (it's the good old DOM vs. SAX issue).

You also can't identify IDs and IDREFs without access to the PSVI, which
opens up that whole PSVI debate, as well. We've mostly avoided use of IDs
and IDREFs in our use of XML, so far, though we do employ IDs that have
significance purely at the application layer (for instance, having
correlation IDs on an XML structure representing a request, which can be
used to correlate the request with a response in a separate document, or
reference the result of processing within another request in the same
document).

<snip/>

> What you present to people in a browser are more likely 
> hierarchical lists
> of length 7 rather than a million rows of relational data. I 
> don't know
> anybody who reviews a million rows of data to make a decision. A
> hierarchical report analysing and aggregating this million 
> rows of into
> different views is way more interesting. That's why I think, 
> even if the
> underlying data storage is not hierarchical, that the 
> hierarchical model is
> well suited for reporting and presentation of data to human beings.

Yes, I agree. And there are relatively few real world users who want to see
raw data in a relational structure. Real world applications hide the
underlying format and present the data in a manner that makes sense to the
user, or they don't get used. That's true with data warehousing efforts, as
well. Most users don't want data; they want information in the form of
documents -- reports, graphs, charts, etc. The most successful data
warehousing projects I've seen offered access to repositories of such
documents that were generated from the underlying warehouse, and provided an
intuitive navigational interface that presented the data as
multidimensional, not relational. And a multidimensional view flattened into
a document or navigational user interface becomes simply a tree, though the
hierarchy can be inverted or otherwise changed based on the order in which
one drills down through the dimensions.