Title: comments on use case #2, and on termination of test case
Mike:
That would be
satisfying: as long as we have a way to exit from a thread (abort)
without
exiting from
the test case.
The only
argument I'd have against your position, is that it is not always on the
"intuiton" side
(e.g. you
spawn an error-detection thread that will explicitly exit on failure as soon as
an error message is received (<WhenTrue><Exit>Fail...) , but
if you forget the <WhenFalse>Continue,
- which
additionally may sound odd if at the end of a thread - then this thread wil
still Fail the entire test case...)
So we have
now 5 possible actions to follow a Test Assertion result (either True or
False):
- continue
(or Continue_Thread?) (default for Assertion=True)
- abort (or
Abort_Thread?)
- exit Fail
(default for Assertion=False)
- exit
Pass
- exit
Undetermined
Am I
right?
Additionally:
- if the test
case terminates its execution (all threads?, or just main thread?) without
exiting either on Fail or Undetermined, then it Passes.
- If a thread
aborts, its container thread (or caller thread) aborts.
- if main
thread aborts: test case Fails? (abort is a logical meaning) or just
Undetermined?
Jacques
-----Original Message-----
From: Michael Kass
[mailto:]
Sent: Monday, August 23, 2004 8:54
PM
To: Jacques Durand; ebXML IIC - main list (E-mail)
(E-mail)
Cc: ;
Subject: Re: [ebxml-iic] comments on use
case #2, and on termination of test case
Jacques,
Below are some of my thoughts
regarding "abort". I think that it is a useful and necessary
function for concurrent Threads
that will later be "orJoined".
I think however, that <Abort> should be
explicitly set by the Test writer, since, in 99% of
<TestAssertion> operations in a typical Test Suite,
a boolean result of "false" for a
<TestAssertion> will signal a defaul exit condition for the Test
Case, with a final result of "fail". This is the case for
ebMS testing, and will likely be the case for
Registry testing as well. BPSS testing will of course have more complex,
concurrent Thread opterations
that would benefit this
functionality. However, making <Abort> the default logical branch
on a failed <TestAssertion> does not represent the typical use
case
in our test scripting.
Also, requiring the test writer to "micro
manage" every <TestAssertion> in the test suite, and explicitly set a
<WhenFalse><Exit>fail</Exit></WhenFalse>
and
<WhenTrue><Continue/></WhenTrue>
is (in my opinion) against the intuitive meaning
of a <TestAsssertion> operation, where "true" = "pass" and "false" =
"fail". Such
default behavior could be "overriden" in the
case of a (rare) "abort" situation. Forcing explicit declarations
of branching will require unnecessary
and labor-intensive micro-scripting for each Test
Case, where more intuitive default behavior rules for <TestAssertion> could handle the majority
of test cases.
I would like to propose keeping
the existing implied <TestAssertion> logic, but adding an explicit
<Abort> option for <TestAssertion> that would let the test
writer abort a
(concurrent) Thread if a particular <TestAssertion> fails.
Comments?
Mike
----- Original Message -----
From:
Jacques
Durand
To: ebXML IIC - main list (E-mail)
(E-mail)
Cc: '' ; ''
Sent: Monday, August 23, 2004 4:01
PM
Subject: [ebxml-iic] comments on use
case #2, and on termination of test case
Mike:
I think we need to tighten the semantics of a Test
Assertion failing without explicit exit statement:
I am concerned that these implicit rules we have will be
confusing on complex test cases that have concurrent threads...
(see my comment in 1.1.4, attached)
I would suggest we consider the following proposal:
- no "fail" outcome is produced unless an explicit fail
exit statement is met during exec.
- no "pass"
outcome unless an explicit pass exit statement is met during exec.
- a failed assertion without explicit exit statement,
by default will "abort" the thread, but just the thread.
- a passed assertion without explicit exit statement, by default
will "continue" the thread.
- when threads are
joined, an aborted thread will automatically cause failure of an and-join
(which aborts the container thread). In case it is an or-join, the aborted
thread will just be ignored by the or-join (the or-join will fail if all
joined threads abort). If a thread that was split but never joined, aborts,
then it just stops and is simply ignored for the rest of the test case exec
and outcome.
- if the Main thread of a test case aborts, the outcome
is "Undetermined" by default (this is the only case of implicit outcome, in
addition to other explicit "Undetermined" outcomes)
Some additional Comments starting p.5 of the attached
doc (mostly, use case #2).
Also, I was searching for the section where we specify
the test step timeout (MaxDuration?) , in the draft spec,
and did not find it (same for the "sleep" statement).
Cheers,
jacques
<<section7.1-JD1_MK_JD2.doc>>
To unsubscribe from this mailing list (and be removed from the roster
of the OASIS TC), go to
http://www.oasis-open.org/apps/org/workgroup/ebxml-iic/members/leave_workgroup.php.
