RE: notes from F2F on conformance - this time with presentationattached!

Title: notes from F2F on conformance - this time with presentation attached!

Hi,

 

    Robert - you have the group's 
thanks. Here are my thoughts :

 

    1.    I will take 
Robert's docs/thoughts and start a very rough conformance document. As Robert 
suggested, we need to have a crisp document by June f2f.

    2.    Robert, I think 
you have the floor on  looking into how other standards handle conformance. 

    3.    We should pick 
a day for con call. What about Wednesdays 4-5 PST ? starting 5/9 
?

 

cheers

            

 

    

  
-----Original Message-----
From: Robert Griffin 
  [mailto:]
Sent: Friday, April 27, 2001 
  9:58 AM
To: ''; 
  ''
Cc: ''; 
  ''
Subject: notes from F2F on conformance - this 
  time with presentation attac hed!

  
hi - 

  
i've attached below the presentation i did on conformance at 
  the F2F. There were a couple of issues that came up during the presentation, 
  or at other times during the F2F:

  
- The most important issue was whether binary 
  compliance/no-compliance was sufficient. The general consensus (in contrast to 
  what Krishna and i suggested in the presentation) was that there needed to be 
  levels or areas of compliance. For example, an implementation that supported 
  the HTTP binding but not the BEEP binding should still be considered compliant 
  with regard to that binding. Similarly, an implementation that consumed 
  assertions but did not produce them could also be considered compliant within 
  its particular area of implementation. 

  
- There was also considerable interest in having at least a 
  strawman for conformance criteria available by next F2F in early June. Several 
  SAML members are already looking at starting reference implementations (or 
  have already begun them, based on earlier submissions to SAML), so we need to 
  start working out the approach to and scope of conformance. 

  
- The question of any OASIS requirements was raised; the only 
  one noted was the expectation of having 3 reference implementations in place 
  before the standard is submitted for a vote (target September).

  
In the presentation, we had identified 3 major areas of work 
  for the subgroup: 1) reviewing the spec for conformance issues etc; 2) writing 
  the conformance part of the spec; 3) working with implementers, including 
  building/coordinating a test harness for compliance. Given the comments in the 
  F2F, i think it's time to start on the second area? Here's a couple of next 
  steps:

  
- Get comparable conformance information for other Oasis 
  standards 
- Sub-group brainstorming on compliance 
  levels, etc 
- Draft a strawman conformance section for 
  the spec, including things like compliance levels, criteria for each level, 
  process for demonstrating compliance.

  
Unless someone else is interested, i'm happy to spend some 
  time next week gathering together conformance stuff from other standards. 
  Krishna, should we try for a sub-group concall sometime like the week after 
  next (that is, week of 7-May)? 

  
regards, 

  
bob