How much run-time validation do you do?

["Roger L. Costello" <costello@mitre.org>]

Hi Folks,

 

Suppose that you have an application which exchanges XML instance documents (instance data) with trading partners.  I'd like to get a feel for how people are addressing these issues:

 

1. Do you validate outgoing instance data?  Do you validate incoming instance data?

 

2. What criteria do you use for deciding whether or not to do run-time validation? 

 

Note: By "validate" I am referring to "XML validation", that is, validate an XML instance document against a schema (DTD, XML Schema, RelaxNG, of Schematron).

 

Also note that I am talking about "run-time validation".  That is, dynamic validation of XML instance data while a system is operational.

 

RUN-TIME VALIDATION SPECTRUM

 

There are two ends of the spectrum with respect to run-time validation:

 

a. Never validate: neither outgoing nor incoming instance data is validated. 

 

b. Always validate: every outgoing and incoming instance is validated.

 

Let's consider each of these:

 

NEVER VALIDATE

 

Suppose that you and your trading partners have agreed to a schema.  And suppose that you tune your application so that it flawlessly generates instance data conforming to the schema.  In this scenario it seems reasonable to skip validating outgoing instance data.  And if your trading partners have similarly tuned their applications then it seems reasonable to skip validating incoming instance data.  (In other words, one-time static XML validation is good enough)

 

>>> MOTIVATION: The motivation for skipping validation is to avoid the performance hit incurred by doing validation.  In scenarios where there are a lot of instances being exchanged then avoiding validation could provide a substantial savings in processing. 

 

Question: Does anyone have performance statistics on validators?

 

ALWAYS VALIDATE

 

Suppose that you are receiving instance data from a variety of trading partners and some of them cannot be trusted to send you conforming data.  In this scenario it seems reasonable to always validate incoming instance data.  And if your application is assembling dynamically created data then it seems reasonable to always validate outgoing instance data.

 

>>> MOTIVATION: The motivation for always validating is to prevent bad data from entering your application, and prevent your application from sending out bad data.

 

Questions:

 

Where does your company stand with respect to run-time validation:

    - Do you validate every outgoing instance?

    - Do you validate every incoming instance?

    - Do you validate only certain outgoing instances?  What criteria do you use for determining which outgoing instances are validated?

    - Do you validate only some of the incoming instances?  What criteria do you use for determining which incoming instances are validated?

 

/Roger