draft presentation on conformance for F2F, for review

Title: notes from conformance subgroup meeting, for review

hi 
-

 

i've 
attached a draft of the 1/2 hour presentation on conformance for the F2F, 
Tuesday morning. Let me know of any 
additions/corrections/suggestions?

 

thanks!

 

bob

 

 

-----Original Message-----
From: Robert Griffin 
[mailto:]
Sent: Tuesday, August 21, 2001 
10:16 AM
To: ; 
''; ''
Subject: notes from 
conformance subgroup meeting

SAML Conformance Sub-group meeting 17-Aug-01 

Attendees: 

        Lynne Rosenthal (NIST) 

        Mark 
Skall (NIST) 
        Gayathri Krishnamurthy (Sena Consulting) 

        Maneesh Bhandan (Sena Consulting) 

        Bob 
Griffin (Entrust) 

Actions: 

        Bob: 
prepare presentation for F2F with issues discussed at today's meeting and 
distribute to sub-group for review.

        Bob: 
continue to define Conformance Program spec test cases and include design-level 
information. 

Summary: 

        We 
quickely reviewed status of conformance work: 

        
        - 
Conformance Clause is in SAML spec, as presented to June F2F 

        
        - 
Conformance Program spec is under development; updated since June F2F with test 
cases 
        
        - 
Schedule for conformance docs and test suite to be determined after August 
F2F 
        Lynne and Mark reviewed the Oasis Conformance TC 
meeting/concall yesterday. 
        
        - 
Documents outlining recommended conformance program for OASIS are being edited 
and reissued, using examples (including from SAML) and with stronger 
recommendations ("shall" rather than "should")

        
        - Would 
be helpful to develop stronger interaction between SAML and ebXML conformance 
groups 
        We then looked at several issues: 

        
        1) Hal 
Lockhart's issue (attached) of whether bindings should be specified as part of 
conformance, or only protocols. Recommendation from sub-group (to be presented 
to TC at F2F) is that conformance test suite be implemented for all bindings in 
the standard, that conformance be defined in terms of protocols, but with 
organization claiming conformance specifying which bindings were 
tested.

        
        2) 
Should test cases include all standard bindings. Recommendation was that we do 
so, so that the test suite can be built with all bindings.

        
        3) How 
should test cases handle optional elements of assertions, protocols and/or 
bindings?  Recommendation was that test suite verify that any 
implementation or application using extensions has correctly specified the 
extension / condition, but that the extension or condition itself will not be 
tested by the standard SAML conformance test suite. Also need to verify with TC 
whether registry for extensions is desirable.

        
        4) How 
should test cases be implemented: for example, by test client inspecting formats 
of produced assertions, by requiring applications/implementations to interact 
with black-box consumer/producer? Recommendation is that this be explored in 
detail with regard to differences between testing profiles and testing 
protocols, particularly with relation to issue raised by Hal.

----------------------------------------------------------------------------------------- 

ISSUE:[MS-3-01: BindingConformance] 

Should protocol bindings be the subject of 
conformance? The bindings sub 
group 
is defining both SAML Bindings and SAML Profiles. It has been proposed 

that both of these would be the subject of 
independent conformance tests. 

The following definitions have been 
proposed: 

SAML Binding: SAML Request/Response Protocol 
messages are mapped onto 
underlying 
communication protocols. (SOAP, BEEP) 

SAML Profile: formats for combining 
assertions with other data objects. 
These objects may be communicated between various system entities. 
This 
might involve intermediate 
parties. 

This suggests that a Profile is a complete 
specification of the SAML aspects 
of 
some use case. It provides all the elements needed to implement a real 

world scenario, including the semantics of 
the various SAML Assertions, 
Requests 
and Responses. 

A Binding would simply specify how SAML 
Assertions, Requests and Responses 
would be carried by some protocol. A Binding might be used as a 
building 
block in one or more 
Profiles, or be used by itself to implement some use 
case not covered by SAML. In the later case, it would 
be necessary for the 
parties involved 
to agree on all aspects of the use case not covered by the 
Binding. 

Thus conformance testing of Bindings might be 
undesirable for two related 
reasons: 

·       The 
number of independent test scenarios is already large. It seems 
undesirable to test something that does not solve a 
complete, real-world 
problem. 

·       Parties 
would be able to claim "SAML Conformance" by conforming to a 
Binding, although they would not be able to actually 
interoperate with 
others in a 
practical situation, except by reference to a private agreement. 

This would likely draw a negative response 
from end users and other 
observers. 

The advantages of testing the conformance of 
Bindings include: 

·       
Simplifying testing procedures when a Binding is used in several 

Profiles that a given party wishes to 
conform to. 

·       Allow 
SAML to be used in scenarios not envisioned by the Profiles. 

This was identified as F2F#3-2. 

        

conform_F2F_23aug01.zip