> it's the reduce functionality in a Hadoop MapReduce task. I'm going to
> split up the huge wikipedia articles revision file (several thousands of
> articles/pages) into
>
> <page>
> ...
> <revision><timestamp>...</timestamp></revision>
> <revision><timestamp>...</timestamp></revision>
> ...
> </page>
>
> chuncks (each page represents one article with several revisions which
> are not entirely sorted), then split into key (timestamp) and values
> (page-fragments) for every page concurrently:
>
> <page>
> <revision><timestamp>...</timestamp></revision>
> </page>
>
> <page>
> <revision><timestamp>...</timestamp></revision>
> </page>
>
Sounds fascinating, and I wish I had time to get involved. It would
certainly be elegant if you could have both the productivity of writing
this declaratively in XSLT and the performance of running it on Hadoop
MapReduce. Intrinsically, the two seem to fit together hand in glove,
but I suspect some engineering effort is needed to make it work.
Michael Kay
Saxonica
