RE: Understanding XML catalogs: what they are, why they are used,and wh

["Costello, Roger L." <costello@mitre.org>]

Hi Folks,

Thank you very much for your feedback last week. Since then I've been getting smart about XML catalogs. Here's what I've learned (every example in this message has been tested and works with SAXON).  /Roger


----------------------------------------------------------------
        WHY USE CATALOGS?
----------------------------------------------------------------
The XML technologies use lots of URLs. While you are in development mode, you might desire to map the URLs to, say, local files. That's what XML catalogs allow you to do, map URLs. And you can do it without impacting your XML documents.

Here's an example which motivates the need for catalogs.

<?xml version="1.0"?>
<BookStore xmlns="http://www.books.org";
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
           xsi:schemaLocation=
                       "http://www.books.org
                        http://www.xfront.com/catalog-test/BookStore.xsd";>


The XML document references an XML Schema file that is remote.  If you are not connected to the Internet then schema validation will fail.  You might instead desire to use a local copy. You can create an XML catalog to instruct the XML Schema validator to use a local copy:

    <uri name="http://www.xfront.com/catalog-test/BookStore.xsd"; 
         uri="BookStore.xsd"/>


This is read as,

    Hey XML Schema validator, when you encounter a reference to this URL:

    http://www.xfront.com/catalog-test/BookStore.xsd

    use this (local) URL instead:

        BookStore.xsd


XML catalogs provide a convenient way to identify where resources--such as XML Schemas--are located. No more hardcoded links.

Norm Walsh has a good description of the value of catalogs:

http://xml.apache.org/commons/components/resolver/resolver-article.html 


----------------------------------------------------------------
        WHAT ARE XML CATALOGS?
----------------------------------------------------------------
An XML catalog is an XML document. OASIS has defined the format of catalogs:

http://www.oasis-open.org/committees/download.php/14809/xml-catalogs.html  


Here's a catalog:

<?xml version="1.0"?>
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">

    <uri name="http://www.xfront.com/catalog-test/BookStore.xsd"; 
         uri="BookStore.xsd"/>

</catalog>


The <uri> element says, 

    Map the link:

        http://www.xfront.com/catalog-test/BookStore.xsd

    to the URL:

       BookStore.xsd


In other words, this instructs the XML Schema validator to use the local copy of BookStore.xsd, rather than the remote copy.


----------------------------------------------------------------
        WHERE ARE CATALOGS USED?
----------------------------------------------------------------
First, wherever links are used, you can use catalogs. 

So, where are links used? Here are some places:

1. In an XML instance document, the value of @schemaLocation is a link to the XML Schema.

2. In an XML Schema document, links are used here:

   <include schemaLocation="... link ..." />

   <import schemaLocation="... link ..."
           namespace="XYZ" />

3. In an XSLT document, links are used here:

   <include href="... link ..." />

   <import href="... link ..." />

   document('... link ...')

   doc ('... link ...')


Second, wherever system IDs and public IDs are used, you can use catalogs. (System ID and public ID is a DTD thing, so if you've never used DTDs then this will be new to you. They are useful, as you'll see.)

This XML document uses a system ID:

<?xml version="1.0"?>
<!DOCTYPE BookStore [
    <!ENTITY book SYSTEM "http://www.xfront.com/catalog-test/book.xml";>
]>
<BookStore xmlns="http://www.books.org";>

        &book;

        <Book>
                <Title>Illusions The Adventures of a Reluctant Messiah</Title>
                <Author>Richard Bach</Author>
                <Date>1977</Date>
                <ISBN>0-440-34319-4</ISBN>
                <Publisher>Dell Publishing Co.</Publisher>
        </Book>
        <Book>
                <Title>The First and Last Freedom</Title>
                <Author>J. Krishnamurti</Author>
                <Date>1954</Date>
                <ISBN>0-06-064831-7</ISBN>
                <Publisher>Harper &amp; Row</Publisher>
        </Book>
</BookStore>


This is called an external entity declaration:

    <!ENTITY book SYSTEM "http://www.xfront.com/catalog-test/book.xml";>


The entity is used (referenced) like this:

&book;


An XML parser will replace the entity reference with the contents of book.xml (which is at a URL).

In the external entity declaration, SYSTEM is a keyword. The combination of SYSTEM and the URL is called a system ID. XML catalogs provide a way to map the system URL to another URL. Here's how:

    <system systemId="http://www.xfront.com/catalog-test/book.xml"; 
            uri="book.xml"/>


Notice that when you want to map a system URL you use the <system> element, whereas when you want to map a link then you use the <uri> element.


----------------------------------------------------------------
        A COMPLETE EXAMPLE
----------------------------------------------------------------
The following XML document references an XML Schema, using @schemaLocation. And it uses a system ID. I want my XML catalog to map the link and the system URL.

<?xml version="1.0"?>
<!DOCTYPE BookStore [
    <!ENTITY book SYSTEM "http://www.xfront.com/catalog-test/book.xml";>
]>
<BookStore xmlns="http://www.books.org";
           xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
           xsi:schemaLocation=
                          "http://www.books.org
                           http://www.xfront.com/catalog-test/BookStore.xsd";>
        &book;
        <Book>
                <Title>Illusions The Adventures of a Reluctant Messiah</Title>
                <Author>Richard Bach</Author>
                <Date>1977</Date>
                <ISBN>0-440-34319-4</ISBN>
                <Publisher>Dell Publishing Co.</Publisher>
        </Book>
        <Book>
                <Title>The First and Last Freedom</Title>
                <Author>J. Krishnamurti</Author>
                <Date>1954</Date>
                <ISBN>0-06-064831-7</ISBN>
                <Publisher>Harper &amp; Row</Publisher>
        </Book>
</BookStore>


Now let's look inside the XML Schema that is being referenced by the XML instance document. It has an <include> element, which contains a link. I want my XML catalog to also map that link.

<?xml version="1.0"?>
<xsd:schema xmlns:xsd="http://www.w3.org/2001/XMLSchema";
            targetNamespace="http://www.books.org";
            xmlns="http://www.books.org";
            elementFormDefault="qualified">

<xsd:include 
     schemaLocation="http://www.xfront.com/catalog-test/Author.xsd"; />

    <xsd:element name="BookStore">
        <xsd:complexType>
            <xsd:sequence>
                <xsd:element name="Book" maxOccurs="unbounded">
                    <xsd:complexType>
                        <xsd:sequence>
                            <xsd:element name="Title" type="xsd:string"/>
                            <xsd:element ref="Author"/>
                            <xsd:element name="Date" type="xsd:string"/>
                            <xsd:element name="ISBN" type="xsd:string"/>
                            <xsd:element name="Publisher" type="xsd:string"/>

                        </xsd:sequence>
                    </xsd:complexType>
                </xsd:element>
            </xsd:sequence>
        </xsd:complexType>
    </xsd:element>

</xsd:schema>


Recap: I want the XML catalog to do 3 mappings:

1. Map the @schemaLocation in the XML instance document.

2. Map the system URL in the XML instance document.

3. Map the link in the <include> element in the XML Schema document.


Here's the XML catalog:

<?xml version="1.0"?>
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">

    <uri name="http://www.xfront.com/catalog-test/BookStore.xsd"; 
         uri="BookStore.xsd"/>


    <system systemId="http://www.xfront.com/catalog-test/book.xml"; 
            uri="book.xml"/>

    <uri name="http://www.xfront.com/catalog-test/Author.xsd"; 
         uri="Author.xsd"/>

</catalog>



----------------------------------------------------------------
        A SECOND COMPLETE EXAMPLE
----------------------------------------------------------------
The following XSLT document has two places where I want to do mappings:

1. It has a system URL that I want mapped.

2. It uses the document() function, which has a link that I want mapped.


<?xml version="1.0"?>
<!DOCTYPE bookstore [
    <!ENTITY discount SYSTEM "http://www.xfront.com/catalog-test/discount.xsl";>
]>
<xsl:transform  xmlns:xsl="http://www.w3.org/1999/XSL/Transform";
                xmlns="http://www.w3.org/1999/xhtml";
                version="2.0">
 
    <xsl:output method="html"/>

    &discount;

    <xsl:template match="/">

        <html xml:lang="en" lang="en">
            <head>
                <title>The ABC Bookstore</title>
            </head>
            <body>
                <h1>
                   <xsl:value-of select="document('http://www.xfront.com/catalog-test/bookstore-name.xml')/bookstore-name" />
                 </h1>
                        
                 <table>
                    <xsl:for-each select="//book">
                        <xsl:sort select="date" order="ascending"/>
                        <tr>
                            <td><xsl:value-of select="title" /></td>
                            <td><xsl:value-of select="author[1]" /></td>
                            <td><xsl:value-of select="date" /></td>
                            <td><xsl:value-of select="ISBN" /></td>
                            <td><xsl:value-of select="publisher" /></td>
                        </tr>
                    </xsl:for-each>
                </table>
            </body>
        </html>

    </xsl:template>

</xsl:transform>


Here's the XML catalog:

<?xml version="1.0"?>
<catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog">

    <system systemId="http://www.xfront.com/catalog-test/discount.xsl"; 
            uri="discount.xsl"/>

    <uri name="http://www.xfront.com/catalog-test/bookstore-name.xml"; 
         uri="bookstore-name.xml"/>

</catalog>


----------------------------------------------------------------
        HOW ARE CATALOGS USED WITH SAXON?
----------------------------------------------------------------
My examples were tested using SAXON. 

If you are using XML catalogs with XSLT transformations then here is how to invoke SAXON:

java -classpath saxon9ee.jar;C:\SAXON-license\;resolver.jar \
     -Dxml.catalog.files=catalog.xml  \
     -Dxml.catalog.verbosity=1  \
     net.sf.saxon.Transform  \
     -r org.apache.xml.resolver.tools.CatalogResolver  \
     -x org.apache.xml.resolver.tools.ResolvingXMLReader  \
     -y org.apache.xml.resolver.tools.ResolvingXMLReader  \
     -o Results.html BookStore.xml BookStore.xsl

Notice the jar file, resolver.jar; you need to download it from here:

http://xerces.apache.org/mirrors.cgi 

The SAXON Wiki has a good description of how to use catalogs with SAXON when doing XSLT transformations:

http://sourceforge.net/apps/mediawiki/saxon/index.php?title=XML_Catalogs 

If you are using XML catalogs with XML Schema validation then here is how to invoke SAXON:

java -classpath saxon9ee.jar;C:\SAXON-license\;resolver.jar \
     -Dxml.catalog.files=catalog.xml  \
     -Dxml.catalog.verbosity=1  \
     com.saxonica.Validate  \
     -r:org.apache.xml.resolver.tools.CatalogResolver \
     -x:org.apache.xml.resolver.tools.ResolvingXMLReader  \
     -t BookStore.xml