[
Lists Home |
Date Index |
Thread Index
]
On Thu, Oct 20, 2005 at 03:19:13PM +0200, Martin Probst wrote:
> Also keep in mind we are still failing ~350 tests of the 7395 we are
> running (making those 95.25% passing tests). A big number (100?) of them
> because we don't support "import schema", but that still leaves over 250
> failures.
A note on the word "failure" here...
There are several possible reasons why an implementation may "fail"
a test, and they do not all reflect badly on the implementation...
(1) a bug in the test (as Mike Kay pointed out)
(2) a bug in the implementation -- OK, this one's not so good :-) but
at least the tests helped to find it
(3) a place where the specification isn't precise enough, so that
different implementations give different results
(4) a place where the spec simply can't be implemented
(5) a bug in the test harness, so that your implementation passes
but it gets recorded incorrectly (or, worse, tests you fail and
incorrectly report as passed). I mention this for completeness.
Items 3 and 4 are the places where we look to see if we (W3C) need
to change the specification as a result of implementation experience.
Those are really places where the *specification* is failing the tests,
not the implementations.
As the specifications move forward towards Recommendations, the
Working Groups involved will try to fix those areas (if there are
any) and I hope will update the tests.
At any rate, I'm really delighted to see people running the tests and
publishing their results!
Liam (W3C staff contact/participant, XML Query WG)
--
Liam Quin, W3C XML Activity Lead, http://www.w3.org/People/Quin/
http://www.holoweb.net/~liam/
|
