Re: [ebxml-iic] Section 7.1 of Test Framework revision.. using Jacques workflow approach (option B)

Title: call tomorrow Wed 8th, 3pm PT

Jacques,

 

   I made changes based on your 
suggestions, also added a few comments/questions ( see attached .. my 
comments/changes in italics )  that we need to resolve.. The 3 big 
items are:

 

    1) Making "mask" the default 
behavior for GetMessage ( I believe this is not necessary, difficult to 
implement, and should not be the default behavior for a Test Driver because it 
assumes too much about the intentions of the test writer ..i.e. that they "don't 
need" any previous messages for their testing)

    2) stepDuration (a Test Driver 
parameter) is set at configuration time, and is used to "timeout" a GetMessage 
(or PutMessage) operation after some reasonable amount of time.  It is not 
meant to be a "timing" mechanism for capturing all received messages withing a 
designated time period, although it could be used that way. Its purpose is to 
give the Test Driver a reasonable amount of time to query the Message Store and 
retrieve messages based upon the XPath Filter.  The use of the 
<Sleep> operator is better suited for synchronizing waits between request 
and response checking.  Actually, a time period like "30seconds" would be 
reasonable for StepDuration ( for our ebMS Test Suite ).  Our Test Driver 
"polls" the MessageStore, and immediately exits from GetMessage() upon 
satisfying its XPath Filter, so if it gets its message in 5 seconds, 
GetMessage() is complete, and the Test Driver executes the next 
operation.   Obviously though, if one does not use <Sleep> 
between a <PutMessage> and a <GetMessage>, then one needs to "tune" 
the StepDuration parameter to allow enough time for the Test Driver to receive a 
response message.  

    3) Do we need to define a 
"stepDurationFail" Test Driver parameter to be used by subsequent TestAssertion 
operations?  We could.  I would argue however that any real "time" 
testing should involve comparison of  message Timestamps, not Test Driver 
clock time ( since network latency, application processing time and test driver 
efficiency are quantities that should not play a role in determining the 
pass/fail of a business process implementation).  Secondly, if we are doing 
to do time measurement using the Test Driver clock alone, then we should "highly 
recommend" the use of <Sleep> to synchronized verification of reception of 
messages, since it does not add unknowns such as effiency of MessageStore 
storage and XPath processer performance ( which could ultimately impact the test 
case result).  StepDuration should not be a toole to accurately measure 
whether a message was received "in time".  

StepDuration is a "tuning" parameter for a Test 
Driver to reasonably set a "timeout" value for GetMessage and PutMessage 
operations.  Nothing more.

  

 

    I agree with the rest of your 
changes.  If we can resolve these last 3 issues, I believe I can finalize 
this section.. and move it to section 4.

 

Cheers,

Mike

 

  
----- Original Message ----- 

  
From: 
  Jacques 
  Durand 

  
To: 'Michael Kass' ; Jacques 
  Durand ; ebXML IIC - main list (E-mail) 
  (E-mail) 

  
Sent: Friday, September 10, 2004 7:33 
  PM

  
Subject: RE: [ebxml-iic] Section 7.1 of 
  Test Framework revision.. using Jacques workflow approach (option B)

  

  
Mke:

  
 

  
agree that 
  a Section #4 in Part 1 should be the place to put the sample scripts (4.1). 
  

  
I'd put 
  "Executing Test Cases" even after the scripts (in 4.2).

  
 

  
More 
  comments on the use cases attached (tagged <JD5>)

  
 

  
Jacques

  
 

  
 

  
    
-----Original Message-----
From: Michael Kass 
    [mailto:]
Sent: Friday, September 10, 2004 
    9:50 AM
To: Jacques Durand; ebXML IIC - main list (E-mail) 
    (E-mail)
Subject: Re: [ebxml-iic] Section 7.1 of Test Framework 
    revision.. using Jacques workflow approach (option B)

    
Jacques,

    
 

    
  Please see my comments below regarding 
    placement of section 7.1 into Part 1 ( I have no problems with it.. but 
    believe it would be better suited in 

    
section 4 instread of trying to put it inside 
    section 3 ( Test Framework Components.. where we describe Test Driver and 
    Test Service).  I think it would 

    
be better to have a "section 4 - Test Case 
    execution model".

    
 

    
  Also, please see 2 attached files ( your 
    modifications to section 7.1 and my response (as MK4) , and the resultant 
    "diffs" file. ( I forgot to turn on tracking... so I highlighted 
    the

    
diffs in red).   I agree/chaged most 
    everything based upon your suggestion.

    
 

    
Cheers,

    
Mike

    
 

    
      
----- 

section_7.1_diffs-JD_MK.doc