Re: [kmip-comment] Possible solution to referencing issues

On 2/04/2014 10:45 AM, Patrick Durusau wrote:

    > PS: I thought an interop test
      shows that different implementations can

      > process the test cases successfully. Not so much conformance
      being

      > tested but that different implementations have the same
      reading of the

      > standard. Not exactly the same as testing conformance, which
      is

      > against the standard and not another implementation.

    

    Actually the profiles are a narrowing of the specification. They all
    note the list of clauses within the specification which apply and
    contain a narrowing of those to specific ranges and in addition also
    specify the flow of messages along with the specifically allowed
    variations on each of the messages within those flows. All of these
    are clearly contained in the conformance clauses within each
    profile.

    

    To make it clear - you can conform to the base specification and not
    conform to a specific profile. You can also conform to the base
    specification and not be interoperable given that most
    implementations will only implement a subset of the very broad range
    of operations and objects supported by the specification (and may
    also select different ranges of allowed enumerations and different
    ranges of length values too).

    

    Your concerns raised to date (assuming I'm correctly following them
    and nothing so far has indicated that I've misunderstood) have been
    that you believe that the test cases should be able to be made
    informative without altering the conformance outcome in the
    documents. Unfortunately that isn't the case. The test cases within
    the profiles are explicitly noted as normative (contained within the
    conformance clauses) and the choice to do that was one made with
    careful consideration within the technical committee.

    

    The purpose of the KMIP specification is to deliver interoperable
    key management and we have as a technical committee elected to solve
    the precision issue by stating the clauses within the base
    specification, the test cases themselves, and a precisely specified
    set of variations included within the profiles to achieve that
    common objective - interoperability. That we are using test cases as
    part of the conformance clauses provides the precision required and
    has the added benefit of being explicitly testable. This goes well
    beyond how many other technical committees within OASIS approach
    this.

    

    For reference, KMIP 1.0 did indeed take the approach you are
    suggesting - of having no normative test cases (and in fact there
    were no test cases in any of the normative documents and a separate
    document as a committee note containing informative test cases
    only). The decision within the technical committee (and discussed
    outside the technical committee itself with a broader group) was to
    add normative test cases in specific separate narrower profile
    documents.

    

    A substantial portion of your feedback items have been based on the
    (incorrect) assumption that the test cases are informative, and we
    have provided feedback that the documents themselves make it rather
    clear that they are not. Additionally, if the test cases were made
    informative rather than normative then the documents would require
    substantial changes in order to attempt to address what is covered
    in the test cases.

    

    As a technical committee we will respond to all the public review
    feedback in due course and that response will represent the view
    point of the technical committee as a whole rather than the
    individuals here (including myself) that are simply attempting to
    provide background as to why we as a technical committee elected to
    take the approach that we have. I suggest we provide the technical
    committee with the time in which to perform that task.

    

    Thanks,

    Tim.