Re: XML and unit testing

[David Brownell <david-b@pacbell.net>]

">"  = Lars Marius Garshol
"> |" = David Brownell

> | Another testing framework to think about is Mauve.
> 
> I've had a brief look at Mauve before, but came away thinking it was a
> collection of test cases for the Java core libraries and nothing more.
> Apparently I was too quick. What else does Mauve contain but a
> collection of test cases?

There's the harness in which those test cases are hang, which
is not specific to the "test java.*" subproblem.  I've not used it,
but anything adequate to test N variants of the standard Java
APIs is clearly able to do more than just "unit" testing.


> | I'm most familiar with separate tests, plugging into some sort of
> | testing framework or harness.  The harness automates running lots of
> | tests, setting each one up and collecting results
> | (pass/fail/expected/...) as well as (important!) organizing
> | unexpected results for analysis.
> 
> I've been using JUnit to create a test suite consisting of by now
> about 650 test cases and am not really very satisfied with it. One
> unmet wish is that it should be easy to collect test cases into
> hierarchical groups, and also to decide which groups to run and which
> to skip. I've had to build utilities on top of JUnit to be able to
> achieve this.

Unit testing != system testing != integration testing ... :)

I absolutely agree that a test harness that can't go beyond
"unit" testing isn't much help in any substantial system.


> A third problem is that with JUnit it seems tricky to reuse the same
> set of test cases for different implementations of an interface. We've
> worked around it by using our mechanism for dynamically generating
> test cases, but again the result was less than great.

That's something that any harness configuration mechanism should
support.  It might be easy to "roll your own" support there,
perhaps with something analagous to a system property that's
given as one input to the "run these tests on that" command.

- Dave