Re: [xml-dev] RE: Caution using XML Schema backward- or forward-compatib

["Pete Cordell" <petexmldev@tech-know-ware.com>]

> PRINCIPLE OF WEB SERVICE DATA VERSIONING

I think this works well for _A_ centralized web service.  As you move more 
and more towards a peer-to-peer messaging framework, such as an XML 
equivalent of SMTP or HTTP, I think it works less well.

I would try to design a schema with the intention of it being versionable, 
but accept that the day may come when you have to give up on it and start 
afresh.  The point at which you give up on your initial syntax base would 
depend on how difficult it is to migrate to a new base version.  The 
adoption rate of IPv6 is perhaps one extreme case study of this.

Is there a similarity here with software that, over successive versions gets 
added to and added to, and then finally its decided to do a complete a 
re-write?  Quite often such re-writes are harder than expected, and not 
appreciably cleaner than what they replace!

Happy new year,

Pete Cordell
Codalogic
Visit http://www.codalogic.com/lmx/ for XSD C++ data binding
----- Original Message ----- 
From: "Costello, Roger L." <costello@mitre.org>
To: <xml-dev@lists.xml.org>
Sent: Tuesday, January 08, 2008 5:59 PM
Subject: RE: [xml-dev] RE: Caution using XML Schema backward- or 
forward-compatibility as a versioning strategy for data exchange


I would like to propose the following data versioning principle:

PRINCIPLE OF WEB SERVICE DATA VERSIONING

Don't base your web service data versioning strategy on a data
validation strategy. Rather, base it on business needs.

DISCUSSION

Don't base your versioning strategy on an XML validation strategy, such
as maintaining backward compatible XML Schemas. It's the data that the
web service makes available to its consumers that matters, not how the
schema is designed. Your web service may have a business need to create
a new version in which new elements are added, old elements are
deleted, and existing elements are rearranged ... many of these changes
would not be possible if your versioning strategy were constrained by
some XML validation strategy such as trying to maintain backward
compatible XML Schemas. The only thing that matters is the XML instance
documents that the web service makes available to its consumers. The
design of the XML Schema that the instance documents conform to is
irrelevant.

QUESTIONS

1. Do you agree with this principle?
2. If so, can you provide a name for the principle?  (e.g., the _______
principle)

/Roger


________________________________

From: Fraser Goffin [mailto:goffinf@googlemail.com]
Sent: Tue 1/8/2008 4:00 AM
To: Costello, Roger L.
Cc: xml-dev@lists.xml.org
Subject: Re: [xml-dev] RE: Caution using XML Schema backward- or
forward-compatibility as a versioning strategy for data exchange



Hi Roger,

this thread has certainly highlighted a major difficulty in dealing
with describing semantics particularly to an anonymous user-base and
especially when an initial meaning is subject to change in a new
revision/version. Even when semantics are [apparently] well understood
Michael Kay brought up the idea of 'semantic drift', something I
suspect many of us have experienced. Most of us no doubt have internal
data policies which try to mitigate the 'abuse' of data and attempt to
cleanly separate out each meaning and its associated data, but these
are not fool-proof particularly in the face of external business
services where resources are under external control and influence.

This has been somewhat interesting, but I am still wondering whether
we have really dealt fully with the subject of syntactic version
management ? Can you propose a set of guidelines [straw-man] for
dealing with these apparently simpler changes (e.g. adding, removing,
re-structuring information items both mandatory and optional, and
cover the idea of owner and user extensibility) ??

This is perhaps a more tangible and potentially solvable problem
domain that I would like to see addressed before I get into the more
difficult issues related to semantics.

Regards

Fraser.


On 04/01/2008, Costello, Roger L. <costello@mitre.org> wrote:
>
> Hi Fraser,
>
> > what approaches we can take to a) identify impacts of
> > specific types of changes made to the data and/or behavioral
> > aspects of processing
>
> In the scenario that I have been promoting (a web service is deployed
> and is available to anyone) it is impossible for the web service to
> know what data changes will impact clients, since the clients are
> unknown and what they do with the data is unknown.
>
> Consequently, the web service operates in its own self-interest: when
> there is a business need, a new version of the data is created.
>
> To minimize the impact of new versions on clients, the web service
> publishes a new URL for each new version.  Accordingly, clients can
> update to a new version of the web service when they have the desire
or
> need.
>
> To be responsive to client wishes and to identify new business
> opportunities, the web service makes available a feedback web page to
> its clients.
>
> Advantages:
>
> 1. The web service is completely decoupled from the clients.  The web
> service needs no knowledge of the clients or their processing.
>
> 2. There is no need for the web service to try to "identify impacts
of
> specific types of changes."
>
> 3. Versioning is based on business requirements, not on (XML) data
> validation limitations.
>
> 4. Clients are not impacted by version changes, unless they want to
be.
>
> 5. It's simple.
>
> Disadvantage:
>
> 1. The web service needs to maintain multiple versions.
>
> Thoughts?
>
> /Roger
>
>
> -----Original Message-----
> From: Fraser Goffin [mailto:goffinf@googlemail.com]
> Sent: Friday, January 04, 2008 7:44 AM
> To: xml-dev@lists.xml.org
> Subject: Re: [xml-dev] RE: Caution using XML Schema backward- or
> forward-compatibility as a versioning strategy for data exchange
>
> Roger,
>
> as Noah mentioned above (copied below) it might be useful to look at
> some specific approaches for dealing with versioning for the '4
> shades' of validation processing mentioned. In particular, although
> there has been much discussion about the *problems* of versioning
> i.r.o syntactic and semantic understanding, the thing that probably
> most of us want to get to is what approaches we can take to a)
> identify impacts of specific types of changes made to the data and/or
> behavioral aspects of processing, and b) how to minimise the problem
> (i.e best practices for designing artefacts that encourage the
> required level of compatibility).
>
> Fraser
>
> noah_mendelsohn@us.ibm.com wrote:
> > Anyway, I'd say there are at least four shades of grey to consider:
> >
> > * Content validation that can be implemented in your schema
language
> (the
> > element name is legal, and the content is an integer)
> > * Content validation that your schema language can't handle (the
> number is
> > prime)
> > * Business validation (that looks like a credit card number, but
our
> > records show that the card was stolen, so it's not "valid" for use
in
> a
> > purchasing transaction)
> > * Semantic incompatibility (we used to use the field for an account
> > number, but in Version 2 of the language it identifies a particular
> credit
> > card)
>
>
> On 04/01/2008, Costello, Roger L. <costello@mitre.org> wrote:
> >
> > > What exactly do you mean by validation?
> >
> > Hi Noah.  By "validation" I mean the use of any XML-based
validation
> > tools, including grammar-based validation (XML Schema, RELAX NG,
DTD)
> > and rule-based validation (Schematron).  By "processing" I mean
> > everything else that a client does after doing validation.
> >
> > /Roger
> >
> > -----Original Message-----
> > From: noah_mendelsohn@us.ibm.com
[mailto:noah_mendelsohn@us.ibm.com]
> > Sent: Thursday, January 03, 2008 5:56 PM
> > To: Costello, Roger L.
> > Cc: xml-dev@lists.xml.org
> > Subject: RE: [xml-dev] RE: Caution using XML Schema backward- or
> > forward-compatibility as a versioning strategy for data exchange
> >
> > Roger:
> >
> > I think this discussion would converge more quickly if you would
> > rigorously define the terms in the propositions below.  What
exactly
> do
> >
> > you mean by validation, for example?  Let's say I have a purchase
> order
> >
> > document and I:
> >
> > * Use XSD to make sure a credit card number element is in the right
> > place
> > in the document
> > * Use Schematron to make sure the expiration date on it is later
than
> > the
> > order date on some element far away in the same document
> > * Use the Java language to pull the credit card number out of the
XML
> > DOM
> > and make sure that some digits in the number properly checksum [1]
> the
> > others (You could probably do this in SchemaTron with some work, or
> in
> > Schema 1.1 assertions if we allowed them on simple types, but let's
> > assume
> > just for the moment that the checksum required computation beyond
> what
> > the
> > schema languages could do, or that you chose not to mess with
coding
> > the
> > LUHN algorithm in XPath.  See [2] for basic information on credit
> card
> > number checksums.)
> > * Use the Java language to open a database of stolen credit card
> > numbers
> > to ensure that the card is still "valid" and not stolen
> > * Use the Java language to place to the order and send a Web
Services
> > message to bill the card
> >
> > Which of those steps do you define as "validation", and which as
> > "processing"7?  Unless you quite carefully define what you mean by
> > processing and what you mean by validation, then it's hard to
> consider
> > an
> > assertion that:
> >
> > 1. Validating data is different from processing data.
> >
> > Indeed, the assertion may follow from or be contradicted by the
> > definitions that you choose, I would think.  Thanks!
> >
> > Noah
> >
> > [1] http://en.wikipedia.org/wiki/Luhn_algorithm
> > [2] http://en.wikipedia.org/wiki/Credit_card_number
> >
> > --------------------------------------
> > Noah Mendelsohn
> > IBM Corporation
> > One Rogers Street
> > Cambridge, MA 02142
> > 1-617-693-4036
> > --------------------------------------
> >
> >
> >
> >
> >
> >
> >
> >
> > "Costello, Roger L." <costello@mitre.org>
> > 12/28/2007 09:02 AM
> >
> >        To:     <xml-dev@lists.xml.org>
> >        cc:     (bcc: Noah Mendelsohn/Cambridge/IBM)
> >        Subject:        RE: [xml-dev] RE: Caution using XML Schema
> > backward- or forward-compatibility as a versioning strategy for
data
> > exchange
> >
> >
> > Hi Folks,
> >
> > The discussion has been truly excellent.  It has clarified many
> > concepts for me.  Thank you!
> >
> > Below is a summary of my understanding of the key concepts that
have
> > emerged from our discussion.  Do you agree with them?  If not,
which
> > ones do you not agree with?  /Roger
> >
> >
> > RELATIONSHIP BETWEEN DATA PROCESSING, DATA VERSIONING, AND DATA
> > VALIDATION
> >
> > 1. Validating data is different from processing data.
> >
> > 2. Just because an application can validate some data doesn't mean
it
> > can process the data.
> >
> > 2.1 Just because an application can process some data that it
> validated
> > doesn't mean that *any* data it validates can be processed.
> >
> > 3. A backward-compatible XML Schema means that a new version of the
> XML
> > Schema can validate instance documents conforming to an old version
> of
> > the XML Schema.  Consider an application that is designed to
process
> > the old instance documents, and suppose that it has obtained the
new,
> > backward-compatible XML Schema.  Now it can validate both old
> instance
> > documents as well as new instance documents.  However, just because
> it
> > can validate the new instance documents doesn't mean it can process
> > them.
> >
> > 4. A forward-compatible XML Schema means that an old version of the
> XML
> > Schema can validate instance documents conforming to a new version
of
> > the XML Schema.  Consider an application that is designed to
process
> > the old instance documents.  It can validate both old instance
> > documents as well as new instance documents.  However, just because
> it
> > can validate the new instance documents doesn't mean it can process
> > them.
> >
> > The following items are targeted at this scenario: a web service
has
> > unknown clients (anyone can use the service); the data it makes
> > available to clients is described by an XML Schema (identified in a
> > WSDL document) and some English prose (in a web page); periodically
> the
> > data is changed (i.e. new version).  See the Amazon web service for
> an
> > example.
> >
> > 5. Versioning the data made available by the web service based on
> > backward- or forward-compatible XML Schemas imposes severe
> restrictions
> > on the types of changes permitted; these restrictions may not be
> > consistent with the needs of the business (the "business" is all
the
> > technical, political, and managerial stuff that went into funding,
> > creating, deploying, and maintaining the web service).
> >
> > 6. Don't base your web service data versioning strategy on a data
> > validation strategy.  Decouple your data versioning strategy from
> your
> > data validation strategy.
> >
> > 7. Base your web service data versioning strategy on business
needs.
> >
> >
> > NOTES
> >
> > The assertions identify XML Schemas as the validation language, but
> the
> > assertions apply to any validation language, such as RELAX NG, DTD,
> or
> > Schematron.
> >
> >
>
_______________________________________________________________________
> >
> > XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> > to support XML implementation and development. To minimize
> > spam in the archives, you must subscribe before posting.
> >
> > [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> > Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> > subscribe: xml-dev-subscribe@lists.xml.org
> > List archive: http://lists.xml.org/archives/xml-dev/
> > List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> >
> >
> >
> >
> >
>
_______________________________________________________________________
> >
> > XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> > to support XML implementation and development. To minimize
> > spam in the archives, you must subscribe before posting.
> >
> > [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> > Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> > subscribe: xml-dev-subscribe@lists.xml.org
> > List archive: http://lists.xml.org/archives/xml-dev/
> > List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
> >
> >
>
>
_______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>
>
>
_______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>
>