Re: [xml-dev] Scatter/gather pattern [was: New XPipe Presentation Available]

At 00:45 22/02/2002 -0500, Thomas B. Passin wrote:
>What is probably the best known application of a scatter-gather pattern
>occurs, not in markup, not in software engineering, but in physics (or
>electrical engineering depending on how you look at things).  I'm referring
>to the famous Fast Fourier Transform.  The FFT lets you analyze the
>frequency content of a waveform in time O[n log n) instead of O[n^2], which
>a straightforward approach (the Discrete Fourier Transform) needs, where n
>is the number of points in the waveform.
>
>The art is in decomposing the problem and putting it back together, and I'm
>sure that remains a constant over the span of scatter-gather solutions.

Yes.


>Conversely, scatter-gather won't be worth the effort unless it yields
>similar savings.

Experience with Scatter/Gather in XML processing has shown that
the benefits are well worth the effort more often than not.
Any developer who has chucked an XSLT/DOM solution in favour
of a SAX/Lexical solution because of memory constraints should
consider Scatter/Gather rather than a rewrite.

The challenge for XPipe is to make the application of the scatter/gather
pattern a no-brainer. We are working on that bit. (And also actively
seeking discussion on the idea of document fulcra and how best
to find these animals in XML corpora.)

Sean