xml-dev - XML Memory Requirements (was Re: Feeling good about SML)

[David Megginson <david@megginson.com>]

"Don Park" <donpark@docuverse.com> writes:

> Still worried I am afraid because it is not just the
> size of the parser and applications but how much memory
> they need to run.  While I am not expecting the size and
> speed differences to exceed 50%

Here are the significant dynamic memory requirements for a conformant,
non-validating XML 1.0 processor that uses an event-based interface:

a) a dynamic buffer for reading names -- its maximum length will be
   the length of the longest name (element name, attribute name, or PI
   target) in the document;

b) a buffer for reading a chunk of data or an attribute value (with
   SAX, its maximum length will be the length of the longest attribute
   value in the document);

c) a stack of the names of currently-open elements so that you can
   match start/end tags; its maximum size will be the depth of
   element nesting in the document (usually under 20 in my
   experience);

d) a map of declared default attribute values;

e) an array of attribute names seen while each start tag is being
   processed (it can be freed as soon as the start-element event has
   been reported); its maximum size will be the greatest number of
   attributes (defaulted and specified) for a single element (usually
   under 10 in my experience, maybe 20 with TEI);

f) a whole bunch of big tables or sets of start/end values for
   checking which Unicode characters are allowed in which contexts;
   and

g) a stack or state variable keeping track of what's currently being
   parsed (i.e. "in comment" or "looking for name").

Note that supporting PIs and comments doesn't really add anything to
the memory requirements, since you can use the same buffers that you
use to read names and character data.

If SML eliminated attributes altogether, you could make (b) a little
shorter (say, 512 characters max), though few documents have attribute
values longer than 512 characters and you could eliminate (d), which
might save 2K or so, and (e) which might save another 1K or so for a
document that really beats up on attributes, but you'd have to add a
bit to (c) for the new elements required to replace the attributes --
let's say, then a 3K saving in memory consumption for processing a
large document if attributes are forbidden.

>, every little bit counts
> so it is a matter of benefit/sacrifice ratio.  Note that
> I also do not believe this is the primary justification
> for SML.

Good, because unless I've missed something obvious, a 3KB memory
saving probably lands on the wrong side of that ratio.

Again, the big problem in creating a tiny XML processor is the
required error reporting for illegal characters in names, attribute
values, etc. -- if you build lookup tables, you're looking at an
enormous amount of memory for each table.


All the best,


David

-- 
David Megginson                 david@megginson.com
           http://www.megginson.com/

xml-dev: A list for W3C XML Developers. To post, mailto:xml-dev@ic.ac.uk
Archived as: http://www.lists.ic.ac.uk/hypermail/xml-dev/ and on CD-ROM/ISBN 981-02-3594-1
To unsubscribe, mailto:majordomo@ic.ac.uk the following message;
unsubscribe xml-dev
To subscribe to the digests, mailto:majordomo@ic.ac.uk the following message;
subscribe xml-dev-digest
List coordinator, Henry Rzepa (mailto:rzepa@ic.ac.uk)