RE: [docbook-apps] Help needed testing CJK search support in webhelp

Hi Ann-Marie and other interested DocBook
users,

I’ve posted some sample content in
German, French, and Chinese from the Madriva documentation that Camille suggested:

 

http://www.thingbag.net/docbook/gsoc2010/sample-de/content/ch01.html

http://www.thingbag.net/docbook/gsoc2010/sample-fr/content/ch01.html

http://www.thingbag.net/docbook/gsoc2010/sample-zh/content/ch01.html
(Chinese content is beneath the headings)

 

The stemming and tokenization appears to
me to be working. The one quirk I notice is in CJK content: if less than the
full query string is matched, then search highlighting doesn’t work for
those hits. This is a fairly minor annoyance. 

 

Please take a looks and let me know if you
see any problems. We still need to demo the case where we don’t have a
stemmer but we’re working on a bug related to that situation.

Thanks,

David

From: Camille B嶲nis [mailto:] 

Sent: Friday, August 13, 2010 2:16
AM

To: 

Subject: Re: [docbook-apps] Help
needed testing CJK search support in webhelp

 

Hi all,

you can find DocBook files in many languages in the Mandriva Linux
Documentation SVN at http://svn.mandriva.com/cgi-bin/viewvc.cgi/doc/MandrivaLinux/trunk/validated/content/Distrib/

HTH.

Camille.

On 12/08/2010 20:20, Ann-Marie Horcher wrote: 

I am bi-lingual, and would be able to test the
German.  I do not have large docbook files to provide, and would not have
time to produce them before deadline.  (Still completing my documentation
and cleaning up final bugs)

 

I read French, but my Chinese is only
conversational.  

 

On Thu, Aug 12, 2010 at 1:53 PM, Kasun Gajasinghe <>
wrote:

 

On 12 Aug 2010, at 09:17 PM, Ann-Marie Horcher <>
wrote:

I am one of the other GSOC students.  I would be
happy to help my compatriot with testing.

 

Hi Ann,

Thank you very much for your kindness. 

Mainly we are in need of verifying the search results
for languages other than English. Currently webhelp has extensive support for
English, French, German and CJK languages. As both David and I not familiar with
these languages, it is little hard to verify the search output.

 

And we need to verify that build process we specified
in the doc is precise, and easy to follow. 

If you can try to build the webhelp and make sure it
works perfectly with one of *your* docbook XML file, it is greatly appreciated.

And if you are familiar with one of these languages I
stated above, and have docbook files to test them, it would be great.

 

Any feedback about this is welcome!

 

Ann, I hope you did a great work for this summer, and
best of luck for your project! :)

 

David, if you can find some docbook files which
doesn't have any confidential issues, please send them to the list.

 

Regards,

Kasun Gajasinghe 

 

On Thu, Aug 12, 2010 at 10:56 AM, Cramer, David W
(David) <> wrote:

Hi Robert,

Kasun knows more about the details of the stemmer, but I can point you to the
documentation for the porter stemmer we used:

http://snowball.tartarus.org/algorithms/porter/stemmer.html

Currently, English, French, and German are supported.

You are correct search does not support wildcards in searches, and I don't
believe that the algorithm would return results with the same base but
different prefixes (i.e. searching for "inhibit" won't show pages
with "exhibit"), but I think that's normal for any search engine.

I'll add "Support wildcards in query string" to the list of future
features. I thought I had added it there already but I see now that it's not
listed.

Thanks,

David

-----Original Message-----

From: Robert Fekete [mailto:]

Sent: Thursday, August 12, 2010 7:14 AM

To: 

Subject: Re: [docbook-apps] Help needed testing CJK search support in webhelp

Hi David,

First of all, thank you for both of you for your work, it looks very promising!

I have a few questions about how search and stemming works:

- Is it possible to add partial matches to the search results? For example, now

if you search for install, installing, or installed, the same results are

returned (correctly), because these words all come from install. But if you

don't type the entire word (say, only 'inst'), there aren't any results.

- Am I right that the search engine does prefix-only matches? (nstall, *nstall,

etc. does not work)

Regards,

Robert

---------------------------------------------------------------------

To unsubscribe, e-mail: 

For additional commands, e-mail: 

-- 

Ann-Marie Horcher

-- 

Ann-Marie Horcher