[Summary] Backward and forward compatible schemas ... Relax NG --> Yes

["Costello, Roger L." <costello@mitre.org>]

Thanks everyone for your excellent comments!  

Special thanks to David Orchard.  David, I read your article.  I now
understand how to implement backward and forward compatible schemas
using XML Schema.  The technique you illustrate is very powerful.

The following shows the technique applied to the example I gave in my
original message.

Example of Backward and Forward Compatibility for Three Versions of a
Schema

Suppose there are three applications - app1, app2, app3.  app1 is
designed to produce and consume the version #1 Book schema.  app2 is
designed to produce and consume the version #2 Book schema.  app3 is
designed to produce and consume the version #3 Book schema.  The
technique enables app1 to process XML instances from app2 or app3; app2
can process XML instances from app1 or app3; and app3 can process XML
instances from app1 or app2.  Nice!

Here's how to design the schemas to support backward and forward
compatibility:

The version #1 Book schema creates an optional <Other> element into
which arbitrary new elements can be placed:

<element name="Book">
    <complexType>
        <sequence>
            <element name="Title" type="string"/>
            <element name="Author" type="string"/>
            <element name="Date" type="date"/>
            <element name="ISBN" type="string"/>
            <element name="Publisher" type="string"/>
            <element name="Other" minOccurs="0">
                <complexType>
                    <sequence>
                        <any minOccurs="0" maxOccurs="unbounded"
processContents="lax"/>
                    </sequence>
                </complexType>
            </element>
        </sequence>
    </complexType>
</element> 

The contents of Book is: Title, Author, Date, ISBN, Publisher and
Other, which can contain anything.

Here's a sample XML instance:

<Book>
    <Title>My Life and Times</Title>
    <Author>Paul McCartney</Author>
    <Date>1998</Date>
    <ISBN>1-56592-235-2</ISBN>
    <Publisher>McMillan Publishing</Publisher>
</Book>

Time elapses ... It is decided to update the Book schema.  We want to
add a <NumPages> element, without breaking old applications (version #1
applications).

Inside the <Other> element add a declaration for <NumPages> and add
another (nested) <Other> element:

<element name="Book">
    <complexType>
        <sequence>
            <element name="Title" type="string"/>
            <element name="Author" type="string"/>
            <element name="Date" type="date"/>
            <element name="ISBN" type="string"/>
            <element name="Publisher" type="string"/>
            <element name="Other" minOccurs="0">
                <complexType>
                    <sequence>
                        <element name="NumPages"
type="nonNegativeInteger"/>
                        <element name="Other" minOccurs="0">
                            <complexType>
                                <sequence>
                                    <any minOccurs="0"
maxOccurs="unbounded" processContents="lax"/>
                                </sequence>
                            </complexType>
                        </element>
                    </sequence>
                </complexType>
            </element>
        </sequence>
    </complexType>
</element>

Now the contents of Book is: Title, Author, Date, ISBN, Publisher,
Other.  Inside Other is NumPages and another Other, which can contain
anything.

Here's a sample XML instance:

<Book>
    <Title>My Life and Times</Title>
    <Author>Paul McCartney</Author>
    <Date>1998</Date>
    <ISBN>1-56592-235-2</ISBN>
    <Publisher>McMillan Publishing</Publisher>
    <Other>
        <NumPages>345</NumPages>
    </Other>
</Book>

This instance will validate against the version #1 schema as well as
the version #2 schema.

Further, the version #1 instance shown above will validate against the
new schema.

Time elapses ... It is decided to update the Book schema again.  We
want to add a <Hardcover> element, without breaking old applications
(version #1 or version #2 applications).

Inside the nested <Other> element add a declaration for <Hardcover> and
add another (nested) <Other> element:

<element name="Book">
    <complexType>
        <sequence>
            <element name="Title" type="string"/>
            <element name="Author" type="string"/>
            <element name="Date" type="date"/>
            <element name="ISBN" type="string"/>
            <element name="Publisher" type="string"/>
            <element name="Other" minOccurs="0">
                <complexType>
                    <sequence>
                        <element name="NumPages"
type="nonNegativeInteger"/>
                        <element name="Other" minOccurs="0">
                            <complexType>
                                <sequence>
                                    <element name="Hardcover"
type="boolean"/>
                                    <element name="Other"
minOccurs="0">
                                        <complexType>
                                            <sequence>
                                                <any minOccurs="0"
maxOccurs="unbounded" processContents="lax"/>
                                            </sequence>
                                        </complexType>
                                    </element>
                                </sequence>
                            </complexType>
                        </element>
                    </sequence>
                </complexType>
            </element>
        </sequence>
    </complexType>
</element>

Now the contents of Book is: Title, Author, Date, ISBN, Publisher,
Other.  Inside Other is NumPages and another Other. Inside the Other is
Hardcover and a third Other, which can contain anything.

Here's a sample XML instance:

<Book>
    <Title>My Life and Times</Title>
    <Author>Paul McCartney</Author>
    <Date>1998</Date>
    <ISBN>1-56592-235-2</ISBN>
    <Publisher>McMillan Publishing</Publisher>
    <Other>
        <NumPages>345</NumPages>
        <Other>
            <Hardcover>true</Hardcover>
        </Other>
    </Other>
</Book>

This instance will validate against the version #1 schema as well as
the version #2 schema as well as the version #3 schema.

In fact, all instances will validate against all schemas.  There is
backward and forward compatibility among all schema versions!

If you would like to see the complete schemas and XML instances, here
they are:

Version #1 Schema:
http://www.xfront.com/backward-forward-compatibility/BookStore.xsd

Version #1 XML instance:
http://www.xfront.com/backward-forward-compatibility/BookStore.xml

Version #2 Schema:
http://www.xfront.com/backward-forward-compatibility/BookStore_v2.xsd

Version #2 XML instance:
http://www.xfront.com/backward-forward-compatibility/BookStore_v2.xml

Version #3 Schema:
http://www.xfront.com/backward-forward-compatibility/BookStore_v3.xsd

Version #3 XML instance:
http://www.xfront.com/backward-forward-compatibility/BookStore_v3.xml

Notice that successive schema versions "added" new elements.  Is there
a way for successive versions to remove elements and still remain
backward and forward compatible?

/Roger