xml-dev - Re: merging multiple XML documents

[Rick JELLIFFE <ricko@geotempo.com>]

"Kaszubski, William J." wrote:
> 
> We have a situation where we need to merge several (anywhere from 2 to 100)
> XML documents into one document that is the union of all the documents. All
> the documents are valid (to the same DTD) and well-formed, and each of the
> lowest-level nodes can be uniquely identified by its value and the values of
> all of its ancestors. All of the nodes in the subdocuments all are sorted so
> that they appear in ascending order according to this unique identifier. We
> need to end up with a document that has the same structure as the input
> documents, contains all of the unique nodes from all the input documents,
> and is also sorted according to the unique identifier.
> 
> Obviously , we can parse each one with the DOM, walk through them all
> simultaneously looking for the node that should appear next, and then copy
> one node at a time to yet another DOM instance, but we fear this might be a
> little slow <g>. Any ideas? Could SAX help here? Is this an XSL problem?
> Where should we begin?

You might find you can do this using entities.

 1) Make each element that needs to be sorted into a separate entity 
(either internal or external) declaration.  Use the sort key as the
entity name
(you may have to play with it if the sort key isnt a NMTOKEN).  (You may
find
 OmniMark or Perl good for this.)

 2) Then create a document/file/stream containing references to all
these entities. 
 (You may find OmniMark or Perl or sgrep good for this.)

 3) Sort that (e.g. using GNU sort or perl or anything) 

 4) Include it into a hub document. 

That way you do everything you need on a streaming basis, with no need
to build any
trees or do any DOM work.  You end up with a document like this

<!DOCTYPE xxx SYSTEM "my.dtd"
[
 <!ENTITY aaaaa1 "<yy><mykey>aaaaa1</mykey><mydata>blah</mydata></yy>" >
 <!ENTITY aaaaa2 "<yy><mydata>blah</mydata><mykey>aaaaa2</mykey></yy>" >
 <!ENTITY aaaab8 "<yy><mykey>aaaab8</mykey><mydata>blah</mydata></yy>" >
]
<xxx>
&aaaaa1;
&aaaaa2;
&aaaab8;
</xxx>

If you normalize this (reserialize it) you will get a more
conventional-looking
XML document (since an XML processor does not have to preserve entity
structure, which is just as well).  

This technique is very common in stream-based SGML systems and can be
very
effective.

Rick Jelliffe

***************************************************************************
This is xml-dev, the mailing list for XML developers.
To unsubscribe, mailto:majordomo@xml.org&BODY=unsubscribe%20xml-dev
List archives are available at http://xml.org/archives/xml-dev/
***************************************************************************