Re: [ubl-dev] Code List Value Validation

["Fraser Goffin" <goffinf@googlemail.com>]

Thanks David, I will take a look. As per Stephens comment below, one of the nice things in the UBL proposal is the ability to use subsets (or the same) code list in multiple contexts within the same document.

 

In terms of implementation, I had been thinking about how the code-lists can be stored (genericode files are one proposal for this) and accessed (schemaTron) and, because of the size and volatility of some of our lists, and the need to provide access to many application platforms, and the desire to have a centrally managed source of reference data, ... yadda , yadda. I was thinking about RDBMS + XQuery as well. I guess there are (at least) 2 aspects to this. Firstly the creation of business rules for value-based validation (this is what shemaTron is good at and is central to the UBL use case), and secondly because I have other possible use cases in mind, the implementation of a service for general access to the reference data to providing full lists, subsets, decoding values, and so on ... Does you implementation provide this sort of 'list manager' functionality ?

 

Fraser.

 

On 07/04/06, David RR Webber (XML) <david@drrw.info> wrote:

Fraser,

 

There are two levels to this - the specification and the implementation.

 

I'd be interested in your thoughts on the lookup() functionality in the OASIS CAM system and the jCAM implementation that supports the use of codelists over XML transactions.

 

The current implementation has baseline functionality for managing and using codelists:

 

 http://www.jcam.org.uk

 

Key from the implementers stance is being able to cover off the following:

 

1) Declarative rule values in XML

2) Lookups to SQL table code values

3) Simple inline code alignment (e.g. 1,2,3,4,5 | A,B,C,D,E)

4) Associative code lookups - if A then X,F,J ; if B then X,F etc

5) Callout to Java routines - permits extended calculated lists via multiple

6) Versioning and local extension values

7) Potential for referencing codelists stored in shared registry service

 

Thanks, DW

Chair - OASIS CAM TC.
 

-------- Original Message --------
Subject: [ubl-dev] Code List Value Validation
From: "Fraser Goffin" < goffinf@googlemail.com>
Date: Fri, April 07, 2006 8:09 am
To: "G. Ken Holman" < gkholman@cranesoftwrights.com>
Cc: abcoates@idmm.co.uk, xml-dev@lists.xml.org,
ubl-dev@lists.oasis-open.org

Ken,

its been a while since I have had a chance to catch up, so hi.

I have been re-reading your UBL Code List Value Validation Methodology (v0.4)
again while I've had a few days off (I know I need to get out more :-),
a few questions if I may be so bold :-

1. How has this work been received in UBL. Is it proceeding as planned. Do
you think there will be any statement on adoption of this methodology any
time soon ?

2. A genericode file contains ONE code list at ONE version right ??

3. In the doc you state that when an enumeration appears WITHIN a schema,
TPs may ligitimately operate a subset since all values are valid in the full
set, but they may not add new values. Am I correct in assuming that the same
is true for a code list which is defined by a standards organisation (not
UBL) but which is NOT embedded within schema ?

As you know in my situation we operate code-lists than are defined by a
standards body and in most cases want to use them in all situations where
there are equivalent semantics (including internal app integration) rather
than create alternative bespoke lists.

Problem is, we do sometimes want/need to extend these lists often to provide
higher fidelity mapping to our operational systems. Another example might be
where a code identifies some high level semantic, but we want to be able to
create a bunch of 'sub' codes to provide a more granular view - accepting
that in 2-way translation there will be data loss. Lobbying the standards
body and getting a timely change/addition can be problematic ?  -  anyway -
I digress

4a. For code lists where there is no established [complete and/or
definative] standard or where the semantics and values are TP relationship
specific, the set of permissible values can be extended and/or restricted
from an offered base set (if available - using your
DocumentStatusCodeCodeType example) or the participating organisations can
agree the set of values (and presumably the list ID to be used in XML
instances). Is this correct ?

4b. If I have many TPs and each has a slightly different relationship, might
this cause me to need a separate genericode file for EACH code list that
differs, however slightly, from another ?. Is there a suggested low
maintenance approach to this problem ?

4c. Similarly to (4b.), if a custom code list is shared across service
contracts for multiple TP relationships, but a need arises to create a new
version with [say] some values added or removed, and we need to be able to
operate both versions concurrently for some period of time, does this
require a complete re-statement of the new code list (in a new .gc file)
with a new version number even if the difference is ONE codified value
(added/deleted/changed) amongst a set of 10,000 values ?

This also means that the implementation will repeat a lot of code. I guess I
am wondering whether there is/should be a way of expressing a 'delta' of
values ?

5. Continuing the theme of (4), we have some code lists which are both
highly volatile (values added and deleted (rarely changed) every month) and
are very large (e.g. > 50000 entries). An example is Vehicle Make/Model. Do
you think this approach is suitable for this type of reference data
(multiple 'active' versions, large number of values) ?

6. Can you explain the difference between the UBL 'CodeType' and
'IdentifierType' in terms of what circumstances you would use either ?

If a schema identifies the ListID, but we want to use a different one (to
employ as 'richer set' of values) , how would an industry standard schema
accomodate this possibility such that the schema remains a standard and
unchanged definition of the structural constraints (is this the
CodeType/IdentifierType approach) ?

7. Do you think that a skeleton context association file could be
auto-generated ?

8. Changing tack slightly. I am interested in using genericode files for a
number of purposes including, value-based validation, UI generation (e,g, to
populate UI controls such as list boxes), transcoding between application
specific codes. It would appear from the genericode materials that this
would be feasible, do you agree ? :-

Std Code    Std Desc        Appl'n A Equiv    Appl'n B Equiv    UI Text
(key)                                (key)                 (key)

abc            Std Widget      def
ghi                     Part No 3321-7 (small widget)

9. What is the suggested approach to deal with deprecated code values. Is
this considered as a versioning issue both for standards based code-lists
(embedded in schema or not) and custom code lists ? Should code lists
include validity date/time values or other 'active/deprecated' indicators ?

10. Caller assertion of list version. If there is no matching version is it
best to flag the validation failure (and possibly reject the message) - that
is, 'trust' the caller assertion, or validate against the un-vesioned
complete list (similar point to the one we discussed earlier about whether
to to an xsi:schemaLocation attribute value) ?

11. Devils advocate: Whats the difference in having to distribute the latest
.gc file versus having to use the latest XSD with updated embedded enums ?
(Ok, I think I know the answer to this one, but it would be good to have a
quote from 'championing' designer, for the benefit of my peer group and
sceptical and untrusting bosses :-)

Cheers

Fraser.

Anthony: So that you are aware, I am attempting to stimulate interest in the
use of genericode within the organisation that I work with (a large UK
financial services company) from a number of potential perspectives. One of
these is value-based validation, hence discussions with Ken i.r.o his work
with UBL, but also for a more broadly accessible resource for reference data
used for a variety of purposes such as UI generation, transcoding, etc..