Re: [xml-dev] A SAX TransformerHandler encoding question

[Rick Jelliffe <rjelliffe@allette.com.au>]

Is your problem that the surrogate characters are both serialized to 
their two UTF-8 equivalents, or that there is a read problem (which is 
the reported issue in the link you mention.)

It should not be surprising if Java Characters are serialized 
independently. You may find that there is some normalization library, 
such as ICU, that can help by transcoding through 32 bit characters, but 
I think you have to set your expectations that using the surrogates is 
still, to an extent, pioneering.

Cheers
Rick Jelliffe

Julian Reschke wrote:
> Hi,
>
> I've get some interesting problems with JDK's (1.4 and 1.5) 
> TransformerHandler and surrogate pairs...:
>
> Consider:
>
>   public void testOut() throws Exception {
>     ByteArrayOutputStream out = new ByteArrayOutputStream();
>     SAXTransformerFactory stf = (SAXTransformerFactory) 
> SAXTransformerFactory.newInstance();
>
>     TransformerHandler th = stf.newTransformerHandler();
>
> th.getTransformer().setOutputProperty(OutputKeys.OMIT_XML_DECLARATION, 
> "yes");
>     th.setResult(new StreamResult(out));
>
>     th.startDocument();
>     th.startElement("", "foo", "foo", new AttributesImpl());
>     char c[] = "\udc00\ud800".toCharArray();
>     th.characters(c, 0, c.length);
>     th.endElement("", "foo", "foo");
>     th.endDocument();
>
>     byte bytes[] = out.toByteArray();
>
>     for (int i = 0; i < bytes.length; i++) {
>       System.out.println(i + ": " + bytes[i] + " " + ((char)bytes[i]));
>     }
>   }
>
> This yields:
>
> 0: 60 <
> 1: 102 f
> 2: 111 o
> 3: 111 o
> 4: 62 >
> 5: -19 ?
> 6: -80 ?
> 7: -128 ?
> 8: -19 ?
> 9: -96 ?
> 10: -128 ?
> 11: 60 <
> 12: 47 /
> 13: 102 f
> 14: 111 o
> 15: 111 o
> 16: 62 >
>
> That is, the surrogate pair has been serialized as two separate 
> unicode characters. It seems that this problem is old (see 
> <http://issues.apache.org/jira/browse/XALANJ-2132>), so why does it 
> still occur in recent JDKs?
>
> Best regards, Julian
>
> _______________________________________________________________________
>
> XML-DEV is a publicly archived, unmoderated list hosted by OASIS
> to support XML implementation and development. To minimize
> spam in the archives, you must subscribe before posting.
>
> [Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
> Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
> subscribe: xml-dev-subscribe@lists.xml.org
> List archive: http://lists.xml.org/archives/xml-dev/
> List Guidelines: http://www.oasis-open.org/maillists/guidelines.php
>