Mike:
in-line. ([Jacques2]
This "timeout period" is something that we need to
integrate into our
Test Driver. I was wondering if the CPA as defined would provide
enough
information to the Test Driver ( assuming that it parses some "base"
CPA
( or "mini-cpa" for configuration ) to determine what that "timeout
period" should be before
checking the message queue ( or some persistent storage structure ).
Will the "base CPA" contain all of the required
information for the Test Driver
to compute its own "timeout period" before checking for received
messages, or
will we have to assign some arbitrary value?
[Jacques Durand] I think it is better to have that as a
testcase-specific configuration parameter...
this is really a test driver control issue, only meaningful to the
test driver, and outside the scope of the CPA .
Maybe in the "operating party" column, we could specify some config
parameters for the
operator of this test step, in addition to the operator name (here
"TestDriver"), e.g. "Timeout=60" (in seconds).
The default value could be given at the beginning of the test suite,
in configuration parameters: e.g.
"DefaultTimeout=120".
[MIKE2] - It looks like there are many
possible places to define test case specific parameters. They could
be defined
as attributes in the <TestCase> column ( or element
for implementation ), the <TestStep> column/element, or the
<PutMessage> or
<GetMessage> operations/elements.
Where to put them would depend upon the granularity at which you wish to
define
such parameters ( parameter for the whole test case? for a
single test step? for a single operation? ) I would
think that
Test Driver configuration would be fairly static for test
cases, and for that reason I would suggest making configuration
parameters
attributes for the <TestCase>
column/element.
[Jacques2] would rule out using the "test
step" col, as it is just a logical id for steps. Either "TestCase"
col or "[Operating] party" col, which would treat this info as
config parameter for the operator (test driver). Granularity: would likely
need to be at step level, e.g. your GetMessage step in Case #74 will need
to wait until its timeout is over in order to count all duplicate messages
- so this timeout limit will be set based on #retries and
retryInterval. But that only makes sense for this step... not
for others in the same Case.
We need also to be clear about the meaning of "reaching" the
timeout: for a duplicate -counting step like in
#74, this is a normal termination: no error (the timeout IS the
step completion signal) but for other steps, if completion is not reached
at timeout, thats an error. We may need to specify when the timeout is
actually used to terminate the step, e.g. timeout_isfailure='false' (true
is default).
1) Regarding your other comments for the 3
abstract tests.. I agree that <Condition>
should be changed to <Precondition>. It has a more direct
meaning in a testing sense.
2) I will fix the typos on the "quotes"
3) Regarding either :
(a) Using predifined CPAs for the "configurator" action,
or
(b) using CPA "templates" and manipulating them like any other
message content...
I would favor (b) simply for the expediency. Or at least, I
think that we should leave that possibility open in our implementation
design. That way, if the number of CPA templates becomes
cumbersome, we could treat the CPA
as just another payload, and manipulate that XML payload template
content the same way we would manipulate
an XML payload template for say ... ebXML Registry
testing.
[Jacques ] I would also try to avoid
using Configurator action.... . (Configurator action assumes the MSH is
capable to dynamically
handle new CPAs, which may complicate things API-wise, may not be true of
all MSH, etc.
But we may still assume that all
the CPAs we need are in reasonable number and pre-installed /
accessible...
(that would be the simplest
solution)
Only in case there are too many
of these, (or too many combinations of
CPA attributes to consider
in our test cases) we will have to specify how to generate
non-preinstalled CPAs from a
template - and that could just be an XPath assignment in a sub-operation,
like for header manipulations. In that case only, we would need the
Configurator, to deploy on a the MSH local to this Test Service.
But I'd say we might not need do
that if we don't need more than 20 or 30 CPAs, which is still a reasonable
number of predefined CPAs...
Comments?
[MIKE2] - So what we are talking about
as far as configurating an MSH using CPAs is a "startup" mode, in which
the
candidate MSH starts up, reads a CPA ( or CPA-like config file
) and configures itself. We are assuming no "dynamic" ability
of an MSH to
alter its configuration once started. Which means
that all tests that must be run must assume that particular CPA
configuration.
[Jacques2] well, in the simplest solution
mentioned above, an MSH should be able to access/use
several CPAs at any time, based on the CPAId in
messages.
All these CPAs would be set (and possibly known from MSH) prior to
running the test suite - may need a pre-configuration phase,
but that really depends on the MSH implementation (instead, CPAs could be
preinstalled in a registry, and MSH only needs to access them on demand,
at run-time, and cache them).
At test suite level, we only need to specify a
list of these CPAs (or CPAIds) needed by our test suite.
Each test case will typically use no more than one CPA, but if we need to
switch in the middle, that just means typically sending a message (m1,
CPAId_1) to the Initiator action, such that this action
will generate a message (m2, CPAId_2).
We only need to change the CPAId in the message, for the MSH to
know what to do.
A
CPA would never change, once "installed" and referenced by a
CPAId.
So that is the least constraining way for the MSH... in case we
really need to dynamically craft and add new CPAs, the COnfigurator
service action would be needed. But that makes more assumptions on the
ability of the MSH. I don't believe we need that for
now...
.
That's fine, but it means that when the
Test Driver "selects" test cases to run, it must make sure that the test
identifies itself ( through, say
<TestCase> attributes, as to
what that test case requires in CPA configuration to execute
correctly.
That way, when the Test Driver loads the
<TestCase>, it can compare its known local "CPA profile" against the
profile of the <TestCase> , and
not run that <TestCase> if
it does not have a matching CPA profile.
[Jacques2] wouldn't the CPAId usage resolve this? We will
assume here that all CPAIds are resolvable. We should not have to specify
upfront the CPA for the test case: MSH should resolve based on CPAId of
messages received, or on messages asked to be sent.
This makes a strong
case for defining configuration parameters as attributes at the
<TestCase> level, and those attributes should include CPA
parameter
values that vary from the "base" CPA profile from which most
<TestCase>s will run. Comments?
Cheers,
Jacques
Mike
At 11:57 AM 10/1/2002 -0700, Jacques Durand wrote:
Mike:
review of your 3 test cases - mostly details.
Note: should we set also some timeout parameter for each test case?
or test step?
(e.g. when we count duplicates, and retries, we have to know when to
stop waiting...)
regards,
Jacques
-----Original Message-----
From: Michael Kass [mailto:]
Sent: Monday, September 30, 2002 10:10 AM
To: Jacques Durand; ''
Subject: RE: [ebxml-iic] next call this Monday
To all,
Here is XML representation of the 3 selected
abstract tests. I will
document/comment this for a clearer
description. This is for an initial look at structure and
underlying
object model.
Mike
