RE: [sca-assembly] A New SCA Testing TC?

I was trying to find some terminology where we didn’t dictate
where new types were published (a company web site is as good a place as any)
provided it met certain criteria of availability. As for having exactly the
same terms and conditions, this can’t even be enforced should a new TC get created
to do the work.

 

Martin.

 

From: Patil, Sanjay [mailto:]

Sent: 20 July 2009 20:06

To: Martin Chapman; Mike Edwards; OASIS Assembly

Subject: RE: [sca-assembly] A New SCA Testing TC?

 

 

The spirit of this proposal makes sense to me. However I am not
sure I follow what 'publicly available' means here. Where would
the material for claiming conformance for the new implementation type be
published? Would such material be available under same terms as
applicable for similar material published by the OpenCSA TCs?

 

-- Sanjay

 

From: Martin
Chapman [mailto:] 

Sent: Friday, Jul 17, 2009 5:59 AM

To: 'Mike Edwards'; 'OASIS Assembly'

Subject: RE: [sca-assembly] A New SCA Testing TC?

On the MS/Siemens issue, how about we change the conformance
requirements in the assembly spec to the effect of a) and b) in your email.

 

Currently we say in Section 13.2 

…

     
3.The implementation MUST support and comply with at least one of the OpenCSA
Member Section adopted implementation types.

 

How
about replacing this with (something like):

 

     
3. The implementation MUST support and comply with a publically available
implementation type. A publically available implementation type MUST:

      a) Have
a publicly available version of the test suite written in the implementation
type, available for any party to download, validate and execute

      b) Have
a publicly available document(s) describing the implementation type, which, at
a minimum, describes the mapping from  implementation type artifacts to
the componentType when the implementation artifact is
used as the implementation of an SCA Component.

      Note
that OpenSCA Member Section adopted implementation types fulfill these
requirements.

 

 

 

 

 

From: Mike Edwards
[mailto:] 

Sent: 17 July 2009 10:01

To: OASIS Assembly

Subject: [sca-assembly] A New SCA Testing TC?

 

Folks, 

Bob's points below
really go into the area of the responses that we owe Microsoft and Siemens

relating to
their public review comments on the Assembly spec. 

It is my belief
that the Assembly tests have been structured in such a way that they are
adaptable 

to a any new
future implementation type, although I agree that the TestCases document needs
to 

be more
explicit and normative about which parts of the testcases can be changed and
what 

changes are
allowed. 

The problem
remains however, about how we can ensure that a vendor implementing a new

implementation
type and wanting to claim conformance for their runtime using that
implementation 

type satisfies
2 key requirements: 

a) Have a
publicly available version of the test suite written in the new implementation
type, 

which can be
used by all and which would be subject to scrutiny for the validity of the
translation 

of the
testcases. 

b) Have a
publicly available document describing the new implementation type which at a 

minimum
describes the mapping from implementation type artifacts to the componentType

when the
implementation artifact is used as the implementation of an SCA Component.

So far, the
only route we have of guaranteeing both of these requirements is to require
that 

the new implementation
language is described by a specification developed under the 

aegis of an
OASIS TC. 

I am interested
to hear if there some alternatives available which will provide these
requirements 

while allowing
some alternative approach to the development and publishing of the materials.

I don't think
that a "Testing TC" affects these questions much at all.  I tend
to agree with Bob that 

the value of a
separate Testing TC is questionable and there remains an issue relating to the

implementation
type specific tests that an impl language TC may want to require as part of the

conformance
requirements for their spec.  To me, these tests seem to naturally belong
to the 

language TC
itself.  Having a separate TC with different membership seems like a recipe
for 

conflict and/or
missing testcases.... 

Yours,  Mike.

Strategist - Emerging Technologies, SCA & SDO.

Co Chair OASIS SCA Assembly TC.

IBM Hursley Park, Mail Point 146, Winchester, SO21 2JN, Great Britain.

Phone & FAX: +44-1962-818014    Mobile: +44-7802-467431  

Email:   

 

  
  
From: 

  
  
  
Bob
  Freund <> 

  
 

 

  
  
To: 

  
  
  

  

  
 

 

  
  
Cc: 

  
  
  
OASIS
  Assembly Test <>, OASIS Policy
  <>, OASIS Bindings
  <>, OASIS Java
  <>, OASIS BPEL
  <>, OASIS CPP
  <> 

  
 

 

  
  
Date: 

  
  
  
16/07/2009
  22:03 

  
 

 

  
  
Subject: 

  
  
  
Re:
  [sca-policy] A New SCA Testing TC?

  
 

 

Ashok,

I am somewhat sympathetic to the argument that it ought to be possible
 

for a new language to be brought in to the family, but I am concerned
 

about creating a centralized testing TC.

One main reason is that it is unclear how long such a TC would be  

staffed or its members remain interested.  A new language could come
 

along years from now.

IMO it would be better to describe the language interface tests in a  

language independent manner so that they could be implemented in  

whatever language as may arise in the future.

The problem is the determination of the correct implementation of  

those "meta" tests.  Perhaps there is some technical
solution, but it  

escapes my limited imagination.

an alternate approach might be that the implementors might self- 

certify and be required to publish the tests used as well as the  

results before being able to claim conformance.  The customer's might
 

them be able to judge for themselves the degree of rigor used and thus
 

the quality of the self-certification.

-bob

On Jul 15, 2009, at 4:42 PM, ashok malhotra wrote:

> If you have been following the Assembly Testing work, you know that
 

> the Assembly Test Cases are written in Java.

> Mike is now preparing a BPEL version.

>

> Clearly, the Java test cases test the Java C&I to some extent and
 

> Bryan has raised an issue that asks whether the Assembly and Java
 

> tests should be more cleanly separated, but there are  

> complications.  The Assembly tests need some C&I to test and
some  

> duplication cannot be prevented.  Also, if the Assembly tests and
 

> the language tests were separated, then who adjudicate differences
 

> of opinion and duplication?  Further, if the Assembly tests were
 

> shorn of any language C&I, someone could come along, pass only the
 

> Assembly, Policy and Bindings tests and claim SCA compliance.

>

> So, I'm suggesting we consider forming a new TC that would be  

> responsible for all SCA Testing.

> Before you go "Aaargh!  Not another TC!", please
consider the  

> advantages:

>

> 1. Bryan's issue would be closed as there would be only a single TC
 

> and it would write and manage all the tests, some

> of which would, of course, cover Assembly and one or more language
 

> C&Is.   In fact, thinking about it, many of the tests

> would cover some Assembly features, some Policy, a binding and some
 

> C&I.

>

> 2.. If all the tests were in a single TC there would be no need for
 

> someone else to settle disputes.

>

> 3. If an outside party were to come and claim SCA compliance, the
 

> Testing TC would have the authority to vet their

> tests and say 'yay' or 'nay'.  Or to enforce which tests they
should  

> run.

>

> What do people think?

> -- 

> All the best, Ashok

>

> ---------------------------------------------------------------------

> To unsubscribe from this mail list, you must leave the OASIS TC that

> generates this mail.  Follow this link to all your TCs in OASIS
at:

> https://www.oasis-open.org/apps/org/workgroup/portal/my_workgroups.php

 

Unless
stated otherwise above:

IBM United Kingdom Limited - Registered in England and Wales with number
741598. 

Registered office: PO Box 41, North Harbour, Portsmouth, Hampshire PO6 3AU