Re: [xml-dev] Here's how to process XML documents written in German

["Tony Graham" <tgraham@mentea.net>]

On Wed, January 30, 2013 6:47 pm, Costello, Roger L. wrote:
...
> This XPath expression does the job:
>
> sum(//Posten[@*[normalize-unicode(name(.)) eq
> normalize-unicode('währung')][. eq 'EUR']])
>
> The normalize-unicode() function converts an attribute name into a
> standard, canonical form.
>
> Lesson Learned:
>
> When processing markup with diacritical marks, beware that two characters
> may visually appear the same but inside the computer they are represented
> very differently. Design XPath expressions accordingly -- use
> normalize-unicode() to convert markup into canonical form.

The truism "validate at trust boundaries" comes to mind: if you can't
trust the encoding or normalization form of the XML that you receive, then
normalise it as soon as you receive it so all of your XML is consistent
and you don't have to make your XPaths unreadable.

Your example is much like the example in Section 3.1.1, "Why do we need
character normalization?" [1] of "Character Model for the World Wide Web
1.0: Normalization".  That document discusses the advantages of early or
late normalization as well as more aspects of normalization that most of
us could think of on our own.  Unfortunately its recommendations are in
flux (and have been since May last year), but your scenario would best be
handled by 'late normalization' where you normalize the data after it's
transmitted to you.

Regards,


Tony Graham                                   tgraham@mentea.net
Consultant                                 http://www.mentea.net
Mentea       13 Kelly's Bay Beach, Skerries, Co. Dublin, Ireland
 --  --  --  --  --  --  --  --  --  --  --  --  --  --  --  --
    XML, XSL-FO and XSLT consulting, training and programming

[1] http://www.w3.org/TR/charmod-norm/#sec-WhyNormalization