XML.orgXML.org
FOCUS AREAS |XML-DEV |XML.org DAILY NEWSLINK |REGISTRY |RESOURCES |ABOUT
OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]
Re: [xml-dev] combining XMLEvent lists


> it's the reduce functionality in a Hadoop MapReduce task. I'm going to
> split up the huge wikipedia articles revision file (several thousands of
> articles/pages) into
>
> <page>
>    ...
>    <revision><timestamp>...</timestamp></revision>
>    <revision><timestamp>...</timestamp></revision>
>    ...
> </page>
>
> chuncks (each page represents one article with several revisions which
> are not entirely sorted), then split into key (timestamp) and values
> (page-fragments) for every page concurrently:
>
> <page>
>    <revision><timestamp>...</timestamp></revision>
> </page>
>
> <page>
>    <revision><timestamp>...</timestamp></revision>
> </page>
>

Sounds fascinating, and I wish I had time to get involved. It would 
certainly be elegant if you could have both the productivity of writing 
this declaratively in XSLT and the performance of running it on Hadoop 
MapReduce. Intrinsically, the two seem to fit together hand in glove, 
but I suspect some engineering effort is needed to make it work.

Michael Kay
Saxonica


[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]


News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 1993-2007 XML.org. This site is hosted by OASIS