[ebxml-iic-framework] Test Framework Comments

Well I feel like I'm 
jumping in the middle of a herd of racing elephants - lots of changes guys and 
tracking them down and resolving is challenging for me.

But hopefully my 
comments will apply or have been take care of and my confusions easy to clarify 
:)

 

I will say good job 
on working on so many fronts of the document in rapid sucession. At some point 
we need to roll back up to the overall document and verify all the sections work 
together. I think once it's all folded in and change tracking removed it will 
look quite nice.

 

I've just read over 
and reviewed Version 0.92 (Jan 20, 2002) - apologies if I've missed updates 
since.

 

Some overall 
comments:

 

To driver-mode or 
non-driver mode ?

----------------------------------------------------

The "driver-mode" 
for Test Driver and Test Service is confusing to me.

 

Test Service Driver 
Mode - whether it notifies the Test Driver of test events or not is clear enough 
and sound to me.

 

Test Driver - driver 
mode is confusing - since you would have a TestService in driver mode and could 
have a Test Driver in non-driver mode.

I suggest using 
conformance mode and interop mode instead and make no mention of 
driver/non-driver mode for the Test Driver. The role of the test driver is 
clearer

and we don't re-use 
the drive mode term from the Test Service which has a quite different semantic 
meaning.

 

Though I'm still 
trying to resolve this with Connection and Service mode - I don't know if they 
are the same thing or orthogonal and speak more of the communication between 
Test Driver and Test Service rather the type of testing being 
done.

 

Are we missing a 
section ?

----------------------------------------

We talk about 
General Methodology and we talk about Test Framework components and we talk 
about Test Suite chemas and instances but we have no 
description

of the runtime of 
the testing.

 

What I mean is how 
to trigger the testing to begin with - we have figure 4 we show a "Control 
Interface" but I can't find a description for it.

In other sections we 
talk about Configuration messages and Initiator messages but we don't described 
how that gets called. I guess I'm looking for the ignition key to the this whole 
engine.

 

 

Should we break up 
the actions and messages into 2 channels - driver control 
versus testing ?

=====================================================================

It seems to me there 
are 2 channels in the test framework.

Control: trigger, 
configuration, test result reporting, notifications, .. etc

Test: messages for 
the purpose of asserting test requirements.

 

 

++++++++++++++++++++++++++++++++++++++++++

No onto the nit 
picky stuff :)

 

Header: date says 
April 2002 (not matching document date)

 

101 - I think we 
should come up with a URI placeholder - so when we do an errata sheet the first 
thing we don't need to do is update this document :D 
(Catch-22)

we can always have a 
page at the URL that states no errata has been created yet but in the future one 
could be.

 

194 - brackets 
around the MIME,XML,SOAP 

I believe SOAP with 
Attachements is usally written [SOAPATTACH]

 

Page 10,11 (367-393) 
all end with periods or none (consistency)

 

466-473 all the 
bullets are related to 464-465 - either indent them one more tab or change 
464-465 into a non-bullet phrase so the rest match up to it.

 

Page 17 (Fig 
5)suggest breaking them up into Fig 5a 5b or 5 & 6 and then refering them by 
that in 499--508

 

will make it a lot 
easier to read - I also suggest moving those 2 figures above the paragraphs that 
describe them.

 

545-551 hints a 
triggering from a message but not how the message itself is sent. (goes to my 
overall point above)

 

612 elts => 
elements

 

671-675 
verificationstatus implies that the Action is doing the verifying -  where 
really the Test Driver will be doing that - so really it just needs to send the 
error it trapped back to the test driver. The Driver will know if that is 
correct or not. 

How about 
<testDriverMessage> or <testNotification> or 
<actionResult>

 

same for 697-701, 
727-734

 

829 level and 
expected to be general enough to [allow configuration of] most implementations, 
...

 

907 fig number is 
out of sequence 4 should be 7, fix all subsequent figures

 

1032-1033 Delete 
[The ebXML MS Conformance Test Requirements instance file can be found in 
Appendix E]

Reason: Appendix E 
is firstly Test Report Schema and secondly the Test requirements is in a 
separate doc

 

1156 This document 
will drive the Test Harness ...

reason: fix 
grammar

 

1186 a Test Case are 
either large XML files or non-XML message payloads.

 

1194-1224 missing 
figure number

 

1258 ... in a Test 
Case MUST evaluate to "true" for a Test Case result to 
"pass".

 

1301-1304 Test 
Driver as driver/non-driver with interop and conformance in paranethesis - seems 
clearerd if we get rid of dirver/non-driver terms

 

1351 delete "the" 
between "parsing" and "a"[after parsing a simple Message 
Declaration]

 

1626 - 1640 XML 
attributes are Uper Cased (e.g. Id, Type, Algorithm) which is different than 
used every where else.

suggest consistent 
UpperCase for Elements, camel (sp?) lowerCase first Word upper Case remaining 
for attributes

 

1800 Content-ID 
change to ContentId 

        Content-Location 
ContentLocation

keep it consistent 
with previous Id usage

remove dashes to 
match MimeHeader and in general XML elements

 

1806 Dsign => 
DSign

 

DSign needs 
attributes camelCased as well

 

1948 
requirement

An enumerated list 
of requirement types of either MUST or MAY

 

2118 - 
2135 CPPA=> CPA

 

we aren't talking 
the specification we are talking the actual document

 

btw what is 
[BASECPPA] ?

 

3578 [ebTESTREQ] 
refers to CPPA spec ?

3584 
[ebTESTSUITE] refers to BPSS spec ? 

 

3607 [ebCPP] = 
either [CPPA] or if we stick with [ebCPPA] style than we have to search and 
replace [CPPA] in document and use the reference type here - I guess the 
prefered way is a good ebXML JC question eh ?

 

Good job 
folks

 

Now onto the latest 
interop doc :D