OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help



   Re: XML tools and big documents

[ Lists Home | Date Index | Thread Index ]
  • From: Tyler Baker <tyler@infinet.com>
  • To: Don Park <donpark@quake.net>
  • Date: Thu, 03 Sep 1998 09:17:00 -0400

Don Park wrote:

> >For my implementation, for ot.xml (a 4 meg document) only about 1-2 megs of
> >is used to store the 4 meg file in RAM due to all Names being cached at the
> >parser level.  It also takes only 10-12 seconds with a P-120 running
> Symantec's
> My test results were from running on Atari 800 (just kidding <g>).  My test
> machine is Pentium-133 with JDK 1.1.6 with JIT enabled.  Building DOM is a
> slow process but there are intermediate forms I am investigating which cuts
> down DOM loading drastically.

Most of all of the hard work of building the DOM tree is already done by the
parser.  Right now with the current state of Java, the single most important
thing I have found that you can do to speed up building the DOM tree is to only
allocate memory for container structures like Arrays, Vectors, or other utility
Collection classes as needed.  In other words, an leaf-node should not have any
memory allocated for storing children.

As for the memory issue, I have thought about some sort of LZW compression of all
of the text in a document tree.  This would save a lot of memory, but may slow
down building the DOM tree a bit.  Any ideas on this?

> >JIT for JDK 1.2 b4 to build the entire DOM tree.  For spitting out the DOM
> tree
> >(and normalizing all the Text nodes) it takes about 15-20 seconds of which
> 5
> >seconds is spent normalizing text nodes and most of the rest of this time
> is
> >actually spent in a brute force search and replace method that scans all
> >character data and attribute values and replaces any occurrences of entity
> values
> >with entity names.  This can be very expensive but I know no other way
> around it.
> Why are you normalizing text nodes before writing them out?  Also, blindly
> replacing entity values with entity names is error prone.

In general I agree with this, and this sort of stuff should be done at the
application level.  Nevertheless, the programmer I feel should have a choice
whether to manually do this or let the formatter do all of the hard work.

As for the text nodes, they do not have to be normalized, just that in my tests I
accounted for this since many people will want to normalize the document tree to
make the output look pretty.


xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/
To (un)subscribe, mailto:majordomo@ic.ac.uk the following message;
(un)subscribe xml-dev
To subscribe to the digests, mailto:majordomo@ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa@ic.ac.uk)


News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS