Re: [ebxml-iic] comments on use case #2, and on termination of test case - a few last words

Title: comments on use case #2, and on termination of test case

Jacques,

 

   I missed one of your comments.. please 
see below:

  
----- Original Message ----- 

  
From: 
  Michael 
  Kass 

  
To: Jacques Durand ; ebXML IIC - main list (E-mail) 
  (E-mail) 

  
Cc:  ;  
  

  
Sent: Tuesday, August 24, 2004 4:40 
  PM

  
Subject: Re: [ebxml-iic] comments on use 
  case #2, and on termination of test case

  

  
 

  
    
----- Original Message ----- 

    
From: 
    Jacques 
    Durand 

    
To: 'Michael Kass' ; Jacques 
    Durand ; ebXML IIC - main list (E-mail) 
    (E-mail) 

    
Cc:  ;  
    

    
Sent: Tuesday, August 24, 2004 3:06 
    PM

    
Subject: RE: [ebxml-iic] comments on 
    use case #2, and on termination of test case

    

    
Mike:

    
 

    
That 
    would be satisfying: as long as we have a way to exit from a thread (abort) 
    without

    
exiting 
    from the test case.

    
The only 
    argument I'd have against your position, is that it is not always on the 
    "intuiton" side

    
(e.g. you 
    spawn an error-detection thread that will explicitly exit on failure as soon 
    as an error message is received  (<WhenTrue><Exit>Fail...) 
    , but if you forget the <WhenFalse>Continue,

    
- which 
    additionally may sound odd if at the end of a thread - then this thread wil 
    still Fail the entire test case...)

    
 

    
[MIKE3] - Understood.. this wasn't the best example I could 
    think of.. 

    
 

    
So we 
    have now 5 possible actions to follow a Test Assertion result (either True 
    or False):

    
- 
    continue (or Continue_Thread?) (default for 
    Assertion=True)

    
- abort 
    (or Abort_Thread?)

    
- exit 
    Fail (default for Assertion=False)

    
- exit 
    Pass

    
- exit 
    Undetermined

    
 

    
Am I 
    right?

    
 

    
[MIKE3] - 
    Yes.  We are implmenting now.

    
 

    
Additionally:

    
- if the 
    test case terminates its execution (all threads?, or just main thread?) 
    without exiting either on Fail or Undetermined, then it 
    Passes.

    
[MIKE3] - 
    Add "exception" in there as well and yes

    
[MIKE4] - A test case terminates 
    ONLY when (anywhere in the workflow) it encounters an explicit exit (insde 
    of <TestAssertion>, or an exception condition (system error).  
    

    
All other operators (continue, abort) 
    simply continue or re-route execution.  When a Test Case 
    terminates, all Threads are stopped, and the main Thread (TestCase) sets 
    the

    
final 
    state (pass/fail/undetermined/exception) of the Test 
    Case.

    
 

    
- If a 
    thread aborts, its container thread (or caller thread) 
    aborts.

    
[MIKE3] - 
    No.. why do we need to do this???  Why kill the parent/caller Thread in 
    which you may wish to do your join????  Also, why not let 
    logic

    
contiue 
    to a final resolution?

    
 

    
- if main 
    thread aborts: test case Fails? (abort is a logical meaning) or just 
    Undetermined?

    
[MIKE3]-We'll go with undetermined, and this will only occur 
    at the <TestCase> level ( treating TestCase as the "main" 
    Thread).  It simplifies scripting greatly,

    
achieves 
    the same goal.  Having a "main" Thread is only meaningful from a 
    semantic view..  TestCase is in f act the main 
    Thread.

    
Comments?  ( We need to move quickly on resolution of 
    this)

    
 

    
 

    
Jacques

    
 

    
 

    
 

    
      
-----Original Message-----
From: Michael Kass 
      [mailto:]
Sent: Monday, August 23, 2004 
      8:54 PM
To: Jacques Durand; ebXML IIC - main list (E-mail) 
      (E-mail)
Cc: ; 
      
Subject: Re: [ebxml-iic] comments on 
      use case #2, and on termination of test case

      
Jacques,

      
 

      
 

      
   Below are some of my thoughts 
      regarding "abort".    I think that it is a useful and 
      necessary function for concurrent Threads

      
that will later be "orJoined".

      
 

      
I think however, that <Abort> should be 
      explicitly set by the Test writer, since, in  99% of 
      <TestAssertion> operations in a typical Test Suite,

      
a boolean result of "false" for a 
      <TestAssertion> will signal a defaul exit  condition for the 
      Test Case, with a final result of "fail".  This is the case 
      for

      
ebMS testing, and will likely be the case for 
      Registry testing as well. BPSS testing will of course have more complex, 
      concurrent Thread opterations

      
that would benefit this 
      functionality.  However, making <Abort> the default logical 
      branch on a failed <TestAssertion> does not represent the typical 
      use case

      
in our test scripting.

      
 

      
  Also, requiring the test writer to 
      "micro manage" every <TestAssertion> in the test suite, and 
      explicitly set a 
      <WhenFalse><Exit>fail</Exit></WhenFalse>

      
and 
      <WhenTrue><Continue/></WhenTrue>

      
is (in my opinion) against the intuitive 
      meaning of a <TestAsssertion> operation, where "true" = "pass" and 
      "false" = "fail".   Such

      
default behavior could be "overriden" in 
      the case of a (rare) "abort" situation.  Forcing explicit 
      declarations of branching will require unnecessary 

      
and labor-intensive micro-scripting for each 
      Test Case, where more intuitive default behavior rules for <TestAssertion> could handle the 
      majority

      
of test cases.

      
 

      
  I would like to propose 
      keeping the existing implied <TestAssertion> logic, but adding an 
      explicit <Abort> option for <TestAssertion> that would let the 
      test writer abort a

      
(concurrent) Thread if a particular <TestAssertion> 
fails.

      
 

      
 

      
Comments?

      
 

      
Mike

      
 

      
 

      
----- Original Message ----- 

      
        
From: 
        Jacques Durand 

        
To: ebXML IIC - main list 
        (E-mail) (E-mail) 

        
Cc: '' ; '' 
        

        
Sent: Monday, August 23, 2004 4:01 
        PM

        
Subject: [ebxml-iic] comments on 
        use case #2, and on termination of test case

        

        
Mike: 

        
I think we need to tighten the semantics of a Test 
        Assertion failing without explicit exit statement: 
I am concerned that these implicit rules we have will be 
        confusing on complex test cases that have concurrent 
        threads...

        
(see my comment in 1.1.4, attached) 
I would suggest we consider the following proposal: 
        
- no "fail" outcome is produced unless an explicit 
        fail exit statement is met during exec. 
- no 
        "pass" outcome unless an explicit pass exit statement is met during 
        exec. 
- a failed assertion without explicit 
        exit statement, by default will "abort" the thread, but just the 
        thread. 
- a passed assertion without 
        explicit exit statement, by default will "continue" the thread. 
        
- when threads are joined, an aborted thread will 
        automatically cause failure of an and-join (which aborts the container 
        thread). In case it is an or-join, the aborted thread will just be 
        ignored by the or-join (the or-join will fail if all joined threads 
        abort). If a thread that was split but never joined, aborts, then it 
        just stops and is simply ignored for the rest of the test case exec and 
        outcome. 

        
- if the Main thread of a test case aborts, the 
        outcome is "Undetermined" by default (this is the only case of implicit 
        outcome, in addition to other explicit "Undetermined" 
        outcomes)

        
Some additional Comments starting p.5 of the 
        attached doc (mostly, use case #2). 

        
Also, I was searching for the section where we 
        specify the test step timeout (MaxDuration?) , in the draft spec, 
        
and did not find it (same for the "sleep" 
        statement). 

        
Cheers, 

        
jacques 
<<section7.1-JD1_MK_JD2.doc>> 

        

        

        

To unsubscribe from this mailing list (and be removed from the 
        roster of the OASIS TC), go to 
        http://www.oasis-open.org/apps/org/workgroup/ebxml-iic/members/leave_workgroup.php.