Re: [xml-dev] Doing QA on an XML parser

[Bjoern Hoehrmann <derhoermi@gmx.net>]

* Frans Englich wrote:
>What I find the tricky part of this, is knowing when one have done it 
>correctly, and in fact have done it at all, and haven't missed anything. What 
>is the correct regression testing/quality assurance to perform on an XML 
>parser? How does one avoid ending up with junk?

>For instance, if one pass W3C's XML Test Suite[1], does that mean anything in 
>practice, or must one resort to some other kind of conformance testing in 
>addition? What do others do?

If you do not pass the test suite that means something. Passing it means
much less since no matter how many individual test cases the suite might
have, your parser might fail to handle the one the suite does not in-
clude. As a simple example, the suite does not have documents > 4GB in
size. The suite is also quite incomplete, it lacks for example these:

  http://lists.w3.org/Archives/Public/public-xml-testsuite/2005Dec/0001.html
  http://lists.w3.org/Archives/Public/public-xml-testsuite/2006Sep/0000.html

both of which triggered bugs in mature and widely deployed processors,
the definition of namespace-wellformedness has actually been changed in
response to some of them, so even if your processor works correctly now,
it might be non-compliant tomorrow. There is not much you can do beyond
ordinary code review and throwing a large test corpus at the parser. You
will probably also need additional suites e.g. to test the UTF-8/16 de-
coders.
-- 
Björn Höhrmann · mailto:bjoern@hoehrmann.de · http://bjoern.hoehrmann.de
Weinh. Str. 22 · Telefon: +49(0)621/4309674 · http://www.bjoernsworld.de
68309 Mannheim · PGP Pub. KeyID: 0xA4357E78 · http://www.websitedev.de/