[
Lists Home |
Date Index |
Thread Index
]
jcowan@reutershealth.com wrote:
> This should upset no-one, because some real characters may require
>
>>more than one Unicode "character" to represent them, anyway.
>>Take Vietnamese, please: if I have a u with a horn accent above plus
>>a dot underneath [1], that is one real character (according to what
>>people think of as characters) but three Unicode characters, 3 UTF-16
>>characters, 6 bytes of storage.
>>
>>
>
>Actually, you can also represent any Vietnamese letter with a single
>Unicode (and UTF-16) character, U+1EF1 in this case.
>
Yes indeed. Developers who are interested in supporting Vietnamese
(etc) in their products
may find it convenient to (do what we do in Topologi, which is to)
normalize text coming
in (open, paste, add accent). The normalization forms recommended by
W3C convert
from multiple combining Unicode characters to single pre-combined forms
where possible:
this means that software written expecting 1 Unicode character = 1
character = 1 glyph
= 1 screen position will work OK.
For Java people, there is a good normalizer implementation at ICU4J:
http://www.icu4j.org
It used to be waaaaaaay to large to be used, but they now have an ant
task so you can
just get a more minimal 355K JAR file with just the normalizer: well
done Mark Davis and team.
Cheers
Rick Jelliffe
|
