Re: [xml-dev] The text illusion

[Ghislain Fourny <g@28.io>]

Hi,

I fully agree with David. It's important to have the stack in mind and to think one level at a time: sequence of bits -> sequence of unicode characters -> XML tree (infoset) -> annotated XML tree (PSVI) -> XPath Data Model, etc. 

To support this idea with an exaggeration, one could even go down to the physical layer with electromagnetic waves transmitting the bits, or atom locations on the disk, and one probably does not want to have to think at this level when validating or querying data.

How to interpret the value of a field is done at the schema level, when you specify that, say, the altitude element has the type xs:integer.

...however :-)

Having said that, of course, the schema used could say instead that the type of the altitude element is xs:base64Binary (with the lexical representation 0000 for example, which can be parsed as an integer or base64, or string, etc). But the decoding, in this case, happens much higher on the stack and has nothing to do with the encoding of the XML file itself.

Kind regards,

Ghislain

Dr. Ghislain Fourny

Chief Scientist

g@28.io

28msec, Inc.

Menlo Park CA | Zurich, Switzerland

http://www.28msec.com/

On Fri, Jul 15, 2016 at 2:05 PM, David Carlisle <d.p.carlisle@gmail.com> wrote:

On 15 July 2016 at 12:52, Costello, Roger L. <costello@mitre.org> wrote:

Hi Folks,

What is the altitude?

           <altitude>0</altitude>

 

The altitude is zero (0), right?

 

No, the altitude is 48. I will now prove it.

 

The content (value) of <altitude> is a single character and that character is represented by hex 30. You can see this if you open the XML in a hex editor:

 

Note: hex 30 equals decimal 48.

So, if we interpret the value of <altitude> as a bunch of bits representing a number, then the answer to the question is this: The altitude is 48.

Q.E.D.

My “proof” is flawed. My proof assumed the bits within <altitude> represents a number. However, bits can represent anything. XML chooses to represent all bits as an encoding of characters. In the ASCII character encoding scheme, the bits 0011 0000 (hex 30) represent the character ‘0’. So the altitude is ‘0’.

Now, you might want to convert the character ‘0’ to a number (cast the value of <altitude> to a number). Such a converter will need to know (a) the bits are an encoding of a character, (b) the bits are an encoding of an ASCII character (and not, say, an EBCDIC character), (c) the character is a character version of a digit (e.g., ‘0’ is a character version of the digit 0), and (d) how to convert the character digit to a numeric digit.

Lesson Learned: When you look at XML you might think you are seeing numbers but you are not. It’s an illusion. You are merely seeing characters.

/Roger

 

It would be wrong to just look at a fragment of the file content in that way and guess the encoding is ascii or otherwise. The interpretation of the xml content should always be as a sequence of unicode code points. The mapping between bits in the file encoding unicode is specified in the xml declaration at the top of the file (or other places such as http headers as specified in the xml rec)

so whether the file is ebcdic encoded or ascii or has &#x30; the interpretation should first be to U+0030  and then to interpret that as the number zero if appropriate.

David