Must Ignore Unkown (retain/discard)

["Fraser Goffin" <goffinf@xxxxxxxxxxx>]

(sorry if this turns up more than once, my subscriptions to various lists 
have been a bit skittish since the server problems)

I sent this one while the servers were having a small tantrum and it
didn't get through, so here is my request for opinions again :-

Many of you will be familiar with this term which is used to describe
an approach to processing XML instances containing allowable content
that MAY be ignored by a receiver if that content is 'not understood'.
'Not understood' is typically related to content in particular
locations (extension points) which is contained in a namespace that is
[foreign] to that of the 'main' schema[s] and which a receiver MAY
have no prior knowledge of. This is a common (ish) approach where a
schema 'owner' wants to allow users of that schema to add arbitrary
(or possibly constrained) content without causing existing
implementations to fail during instance validation (at least if they
are only using standard schema validation capabilities of mainstream
parsers).

David Orchard has written much on this subject (as have a few others)
and also describes 2 variants of the must ignore unknown pattern,
specifically, 'discard' and 'retain'. As I understand it, the former
means that unknown content can be both ignored and discarded (not
passed to upstream processing) without generating and error, and the
latter, that content may be ignored but should *not* be removed. It is
the 'discard' aspect which, when I was discussing the possible use of
this approach recently, that came under some challenge. I would be
interested in this forums view :-)

The, not unsurprising, challenge was/is this :-

in a situation where :-
- message data is captured by some application
- the basic content model for the transaction is defined by a standard
schema to which all participants agree to conform
- the standard schema allows for extensibility at various points so
long as these are defined in a foreign namespace.
- some of the data captured has been specified by only one provide of
the service and that provider has arranged with the application owner
to put that data in the appropriate extensibility area in an agreed
foreign namespace.
- the message (including all extension data) will be sent to *all*
potential service providers of which there may be many
- the service provider who requested the additional data wants to use
the standards based data model *not* create a completely private
schema for this transaction.

so ... what should receivers of the message who do *not* understand
the extension do ?

Are they likely to be obliged (possibly by legal, regulatory, audit,
.. requirements) to retain ALL data that a customer has agreed to send
(perhaps for non repudiation, DPA, or other reasons) regardless of
whether they intend to process it or not. And if so, does that make
it a practical non starter given that the size and content of 'unknown
data' requires them to provide an adequate (and equally unknown)
storage (and retrieval) capability  (at least for those business
transactions to which these sort of obligations might apply) ???

Opinions welcome

Fraser