Caution using XML Schema backward- or forward-compatibility as a version

["Costello, Roger L." <costello@mitre.org>]

Hi Folks,

Designing XML Schemas to be backward- or forward-compatible is a
popular approach to data versioning.  

I think some cautions need to be raised with versioning strategies
based on XML Schema backward- or forward-compatibility.  

Below I list the cautions.  Do you agree with these cautions?  Are
there cautions I have missed? 


SCENARIO

Consider deploying a web service.  Assume the web service has no
knowledge of who its clients are, or how clients use the data they
retrieve from the web service. 

The web service uses an XML Schema to describe the syntax of the data
it exchanges with clients.  

The web service uses the following data versioning strategy: 

    Each new version of the XML Schema is designed to be
forward-compatible. 

Thus a client with an old XML Schema can validate an XML instance
document generated by the web service using a new XML Schema.  


ISSUE

Given the scenario described, what cautions should be raised on the use
of forward-compatible XML Schemas as a versioning strategy for data
exchange?"


NOTE

A versioning strategy based on backward-compatibility has the same
cautions. I will not explicitly mention backward-compatibility in the
rest of this message, but bear in mind that the comments apply to it as
well.


CAUTION #1: JUST BECAUSE A CLIENT CAN VALIDATE THE DATA IT RETRIEVES
DOESN'T MEAN IT CAN PROCESS THE DATA

Consider a client application implemented to process version 1 data
from the web service. 

Suppose the web service changes its XML Schema, in a forward-compatible
fashion.  Will the client application be able to process the new
(version 2) data?

Since the XML Schema is forward-compatible the application will be able
to "validate" the new data.  

But it is not necessarily the case that the application will be able to
"process" the new data.  

Example #1: Suppose in the first version of the XML Schema this
element:
   
    <distance>100</distance>

means "distance from center of town."  Accordingly, the client's
application does calculations based on that meaning.  

In the version 2 data the syntax is changed in a forward-compatible
fashion.  In addition, the semantics of the <distance> element is
changed to "distance from town line."  

The client application will be able to validate the version 2 data, but
the calculations will be incorrect.

Example #2: If the version 1 XML Schema defaults the <distance> units
to miles and the version 2 XML Schema defaults the <distance> units to
kilometers then the data will validate but the client's application
will make incorrect calculations.

Lesson Learned #1: Data may change syntactically in such a way that
validation is not impacted, and yet applications break.  

Lesson Learned #2: Just because an application can validate data
doesn't mean it can process the data.

Lesson Learned #3: Forward-compatible XML Schemas yield increased
validation but not necessarily increased application processing.

Lesson Learned #4: There is no necessary correlation between the
ability to validate data and the ability to process data.

Lesson Learned #5: A versioning strategy must take into account:

    1. Syntactic changes
    2. Relationship changes
    3. Semantic changes 


CAUTION #2: FORWARD-COMPATIBLE CHANGES ARE BASED ON TECHNOLOGY
LIMITATIONS RATHER THAN APPLICATION REQUIREMENTS

Designing a new version of an XML Schema to be forward-compatible with
an old version necessitates that the only changes made in the new
version are "subset" changes, such as:

    - constrain an element's or attribute's datatype 
    - reduce the number of occurrences of an element  
    - eliminate an optional element or attribute  
    - remove an element from a choice  

This is very restrictive.  And to what avail?  Answer: to enable
validation of new XML instance documents against an old XML Schema.
But as described above, just because data can be validated doesn't mean
it can be processed.  

Further, for the scenario we have been considering, the web service has
no idea about how its data will be processed by clients.  Accordingly,
there is no evidence that the additional validation provided by
forward-compatible XML Schemas will help clients.

Lesson Learned #6: A versioning strategy based on forward-compatible
XML Schemas imposes limitations on the types of changes; those
limitations may not be consistent with the actual changes needed by an
application.

Lesson Learned #7: Version data based on data requirements rather than
technology limitations.


QUESTIONS

1. Do you agree with the cautions listed above?  

2. Are there other cautions?

3. Do you agree with the Lessons Learned?

4. Given the scenario described above, is it wise to base a versioning
strategy on forward-compatible XML Schemas?

/Roger