Two tips for dramatically increasing the speed of validating a largenumb

[Roger L Costello <costello@mitre.org>]

Hi Folks,

Note: The following only applies if you are using SAXON for schema validation.

I have over 6,000 XML files that I need to validate against an XML Schema. Initially, I used commands in a batch (.bat) file to loop over all the XML files and call SAXON to validate them. I never got through all 6,000 files because it was taking so long; I estimate it would have taken 3 hours or more. 

I looked on the SAXON web page and found some tips for dramatically increasing the speed of validating a large number of XML documents.

Tip #1: Compile your XML Schema to a Schema Component Model (SCM). Here's how to generate the SCM file:

java com.saxonica.Validate -export:file.scm -xsd:file.xsd

That results in creating file.scm which is a compiled version of file.xsd

You can use file.scm anywhere you would use file.xsd and it will run much faster.

Tip #2: Specify all your XML files to be validated using the glob syntax, *.xml

Suppose all the XML files to be validated are in the folder named instances. Then use this command to validate all of them:

java com.saxonica.Validate -xsd:file.scm instances\*.xml

Notice that that uses the SCM file not the XSD file.

I used those two tips to validate my 6,000 XML files. How long did it take to validate the entire bunch? Answer: Immediate (i.e., way less than one second). So, the time to validate all 6,000 XML files dropped from 3 hours to less than one second. Pretty cool, aye?

/Roger