[
Lists Home |
Date Index |
Thread Index
]
All my word breaking is delegated to a class called (surprise) WordBreaker,
which implements a very simple algorithm that uses Java's
Character.isLetterOrDigit() function to determine where words begin and end.
This works well for Western languages. If you want to optimize for a
non-Western language, you can override WordBreaker and implement word
breaking in whatever way makes sense for your particular language or
languages of interest. That's the theory at any rate ...
Howard
-----Original Message-----
From: Charles White [mailto:chuck@tumeric.net]
Sent: Saturday, December 06, 2003 12:45 AM
To: xml-dev@lists.xml.org; Elliotte Rusty Harold
Subject: Re: [xml-dev] ANN: XQEngine 0.61
Or for Cherokee, for that matter, which is REALLY nasty, even if you're
completely XML 1.0 and Unicode compliant??!!! God forgive us all.
Chuck White
Author, Mastering XSLT, Sybex Books
http://www.javertising.com/webtech
http://www.tumeric.net
----- Original Message -----
From: "Elliotte Rusty Harold" <elharo@metalab.unc.edu>
To: <xml-dev@lists.xml.org>
Sent: Friday, December 05, 2003 8:28 AM
Subject: Re: [xml-dev] ANN: XQEngine 0.61
>
> >f) a new contains-word() function for full-text search
>
>
> I note from your web page that:
>
> The contains-word() function now allows multiple words as arguments.
> These are implicitly anded together, as in either contains-word(
> //elemName, "word1", "word2", "word3" ) or contains-word( //elemName,
> "word1-word2 word3" ) (using internal whitespace and/or punctuation
> as word separators).
>
> This doesn't seem very internationalizable. How does it work with
> languages such as Thai that have non-whitespace based rules for word
> breaking?
>
>
> --
>
> Elliotte Rusty Harold
> elharo@metalab.unc.edu
> Effective XML (Addison-Wesley, 2003)
> http://www.cafeconleche.org/books/effectivexml
>
http://www.amazon.com/exec/obidos/ISBN%3D0321150406/ref%3Dnosim/cafeaulaitA
>
> -----------------------------------------------------------------
> The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
> initiative of OASIS <http://www.oasis-open.org>
>
> The list archives are at http://lists.xml.org/archives/xml-dev/
>
> To subscribe or unsubscribe from this list use the subscription
> manager: <http://lists.xml.org/ob/adm.pl>
>
>
>
-----------------------------------------------------------------
The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
initiative of OASIS <http://www.oasis-open.org>
The list archives are at http://lists.xml.org/archives/xml-dev/
To subscribe or unsubscribe from this list use the subscription
manager: <http://lists.xml.org/ob/adm.pl>
|
