Re: [xml-dev] Schematron tip: Use only one pattern

[Michael Kay <mike@saxonica.com>]

“Why is validation so slow?” I wondered.

I did some digging and discovered the reason for the slowness. Recall that a Schematron schema is converted to an XSLT program. I looked at the XSLT program that was generated for my Schematron schema. Here’s what I found:

   <xsl:template match="/">
      <xsl:apply-templates select="/" mode="M0"/>
      <xsl:apply-templates select="/" mode="M1"/>
      <xsl:apply-templates select="/" mode="M2"/>
      <xsl:apply-templates select="/" mode="M3"/>

      …
      <xsl:apply-templates select="/" mode="M29"/>
   </xsl:template>

 

That says: Start at the top of the XML document and process the entire XML document, applying the template rules with mode M0. Then, start all over at the top of the XML document and process the entire XML document, applying the template rules with mode M1. And so forth. 

 

So, the entire 180 MB XML file gets processed 30 times.

 

Eek!

 

It’s no wonder that validation was so slow.

It's interesting that this should make a big difference, but it's not at all inevitable. I would expect that the actual navigation (selecting child nodes) is very fast compared with the cost of evaluating match patterns and executing the template rules, and since each pass of the document is in a different mode, there should be very little cost for processing nodes that are effectively skipped. So the reasons for the slowness must lie a little deeper than you imagine. A lot will depend on the actual match patterns present in each mode. And of course, on the XSLT processor, since processors are likely to use very different algorithms for trimming the search space for pattern matches.

And going into meta mode, as one does on xml-dev: It's quite common for a community to develop folk wisdom along the lines "don't do X, it's very slow" based on an experience like this that might (or might not!) turn out to have general applicability.

With performance, the devil is always in the detail.

Happy Christmas!

Michael Kay

Saxonica