Re: performance test suite? (was: Re: Using W3C Regular Expressions)

["Al B. Snell" <alaric@alaric-snell.com>]

On Fri, 20 Apr 2001, James Strachan wrote:

> I've not managed to track the whole text-v-binary discussions -  I find it
> hard to find the time to keep up with xml-dev these days :-(.
> Is your intention to build a performance testing framework to demonstrate
> the performance difference of binary v. text XML parsers?

Something like that. The Community is crying out for hard figures rather
than speculation, so when people's implementations are ready and tuned to
the hilt, I'd like to organise a race.

For now, I'm collecting ideas on testing procedures that will ensure
fairness.

> If so, I've been meaning to investigate the relative performances of a
> variety of SAX parsers (crimson, Xerces1/2, Aelfred), both with and without
> validation along with some other configurations like turning on and off of
> String "interning".

Validation on and off - I forgot about that - another criteria for my
list :-)

> It might be a good idea to combine our efforts into the same benchmarking
> suite so (say) it would be easy to compare a binary SAX driver fares against
> the best text SAX drivers etc.
> 
> My idea was to create a SourceForge project for the benchmark suite.
> Does this seem of interest or am I way off where you're at?

Oooh, that'd be sweet. The binary implementations aren't very ready yet,
but the testing tools will be useful for comparing different parsers for
now, and they can be designed to allow plugging in binary formats later
easily (this may come for free if they support multiple implementations
anyway)

> 
> James
> 

ABS

-- 
                               Alaric B. Snell
 http://www.alaric-snell.com/  http://RFC.net/  http://www.warhead.org.uk/
   Any sufficiently advanced technology can be emulated in software