From the charter:
http://www.oasis-open.org/committees/sca-assembly/charter.php#item-4
"The TC shall define concrete exit criteria that include at least
two independent offerings that implement and are compliant with the
all normative portions of specifications and demonstrate
interoperability and portability as appropriate. Note that these are
minimums and that the TC is free to set more stringent criteria. "
The ambiguity of English gets in the way here a little. Does this
mean:
For each normative statement, at least two implementations?
a minimum of two implementations that each fully implement all
normative statements?
I suspect we strongly want to prefer the former. That means we can
potentially rely on lots of implementations that don't claim
conformance to the spec, but do help us to achieve coverage.
What this discussion turns on, then, I think is the meaning of
"implement", and "compliant". Is it sufficient for a vendor to
declare that they've implemented and are in compliance? Or do we
have to "prove" that in some way? I'm satisfied if a vendor that
claims to implement a normative statement.
As for "and demonstrate interoperability and portability", based on
what I understand of the existing test suites, that two
implementations that pass the test suite is a pretty darn good
demonstration, and I wouldn't go further than that. I'm with Jim in
that at some point we have to rely on the court of public opinion.
From a pure "quality of work" perspective, I think if two
implementations pass the test suite, and they comment about the
normative statements not covered by the test suite, then we've
acheived sufficient "code coverage" if you will, of the spec itself.
I've expressed an ongoing concern that we've over-specified parts of
SCA. Certainly, within the bindings TC, some of you have heard me
express that point. To pick on a random point (really, I just looked
at the normative statements and this one caught my eye - hadn't
thought about it before), the "sca-contribution-generated.xml" file
declared for deployment - how is this file an essential part of
testing conformance? Since there's no language that I'm aware of
that *excludes* other files from being processed as part of the
contribution, why couldn't this detail be left to an implementation?
Do we *require* this for the specification?
If a side-effect of the "two implementations" constraint leads us to
de-normalize (de-normativize?) some of the current required
language, that might be a good thing.
I'll reiterate what I said in the call. We should check with the
implementations that we're aware of, and find out which normative
statements they don't believe they've implemented. With that
information in hand, we'll better understand our options for this
question. (And I'm not volunteering to take on this task because I
don't know who to talk to, and I'm not familiar with the existing
test suite.)
-Eric.
On 2/22/11 12:08 PM, Jim Marino wrote:
Hi,
The first bullet makes sense to me. Concerning the second, my
understanding is the TC Test Suite is not normative and the only
thing required for conformance is for the entity responsible for
a particular runtime to declare that it adheres to the normative
portions of the Assembly Specification. In other words, it
should be valid for a runtime to claim conformance that supplies
its own test suite, which may or may not be based on the TC one.
This essentially goes back to using the "court of public
opinion" to judge conformance claims as opposed to something
akin to a TCK.
Jim
On Feb 18, 2011, at 6:19 AM, Mike Edwards wrote:
Folks,
I propose the following as
the Exit
Criteria to be adopted by the SCA Assembly TC:
"The Concrete Exit Criteria
for
the SCA Assembly V1.1 specification are that:
o there shall be 2
independent SCA runtimes
that are compliant with all normative portions of the
specification as described
in Section
12.2 of the SCA Assembly V1.1 specification
o the 2 independent
runtimes shall pass
the Test Suite for SCA Assembly as described in the
document
'TestCases for the SCA
Assembly Model
Version 1.1 Specification' "
I believe that this is
sufficient to
cover:
- the question of
conformance to the
spec
- the issues of portability
and interoperability
since the test suite is a
pretty thorough
examination of conformance to the spec (there are
testcases for
all normative statements
with testable
& observable consequences) and the test suite is also
a test of portability,
since the test suite has an
extensive
set of artifacts that are ported to each runtime and
interoperability is
checked through the use of
Web services
being used between the SCA runtime and a non-SCA (JAXWS)
client.
Yours, Mike
Dr Mike Edwards
Mail
Point 146, Hursley
Park
<Mail Attachment.gif>
STSM
Winchester,
Hants SO21
2JN
SCA & Services
Standards
United
Kingdom
Co-Chair OASIS SCA
Assembly TC
IBM Software Group
Phone:
+44-1962
818014
Mobile:
+44-7802-467431
(274097)
e-mail:
Unless stated
otherwise above:
IBM United Kingdom Limited - Registered in England and
Wales with number
741598.
Registered office: PO Box 41, North Harbour,
Portsmouth, Hampshire PO6
3AU
