OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]
Re: [xml-dev] XML not ideal for Big Data


There've been any number of cases where I've pre-parsed text formats using Perl or similar tools into "raw" XML files and then converted those into something that was actually useful, and I'd say there's probably a significant set of data for which XML is NOT appropriate. I recently worked with someone doing analysis on several TBytes of Bathyspheric data that made FAR more sense to keep in a relational database - huge amounts of data, but relatively few relational bindings, data being processed was principally numeric and so forth. Putting something like that in XML would have been insane, and definitely not worth the effort.

The problem is that people confuse large data sets with complex data, then seem to assume that simply because the use case for the first is not something you'd want to use XML for, that XML is universally bad for the latter as well. That to me seemed to be the thrust of the initial article - that XML is bad because it's not good enough to handle every use case. We've had a decade worth of use cases, I think that anyone who's been working with XML in the field for any length of time should get a sense for where XML is appropriate and where it's not.

I think that XML has gone through Gartner's roller coaster. It went through an adoption and hype phase (I remember it getting silly in 2001) and it went through its nadir (2004-2006). It's a mature technology now. People did try it for a lot of things it was not appropriate for, but frankly, that's okay - good technologies usually tend to be used for sometimes very silly things, and the disasters are usually marked with big "Don't Go There!" signs after the fact; it's how you determine where a tech can be used, and I'm sometimes very surprised at just how pervasive the technology has become.

Kurt Cagle
Managing Editor

On Thu, Sep 3, 2009 at 6:48 PM, Michael Sokolov <sokolov@ifactory.com> wrote:
So many points were made arguing for XML being OK for "big data," many of
them sensible to me.  Just to be clear: I use XML databases day in and day
out, I work with large XML files and it's all just dandy.  I don't think
size is an issue, mostly.

However I think we need to recognize that there is data for which XML was
not designed and is ill-suited.  Binary, numeric data, such as video, images
and audio, to say nothing of scientific data (years of detector readings in
a neutrino decay experiment) is just not the sweet spot for XML.  I searched
for "MP3 to XML converter", but couldn't find anything.  I have to admit I
was surprised: the net is so big that I thought it had finally reached the
stage where enough monkeys typing will have produced absolutely everything.
Maybe my search skills just weren't up to the task.

Now it's hard to tell if his problem fell in the domain for which XML is not
well-suited.  I don't know what the details of the original writer's project
were, but I would tend to want to take him at his word that XML was not the
right choice for his data.  It's certainly possible: there is such a domain.
And genomics data sounds to me pretty unlike documents: it probably wouldn't
pass the smell test that was discussed earlier. XML is not for everything.

As an aside, XML is also not always the right choice for every*one*, either,
regardless of the problem domain.  Even if it might have been possible for
someone else to achieve success with a genomics dataset using XML rather
than CSV and perl or whatever he used, I think his point is still valid.  He
doesn't want to spend time learning XML technologies: he just want to get
the project done.  So if learning XML (document format, query language,
database technology, etc) was too hard and he managed to find success some
other way, I don't think that's any reason to disparage him.  He found a
tool that suited his purposes and the context in which he was working.

Last point: the only reason people write articles like his is that XML was
touted as the everything/everywhere solution for so long. For me it's still
about (human-readable) documents and data interchange, primarily.  I'm
curious whether there is agreement on that, or do folks see other broad
areas where XML is beneficial?


> -----Original Message-----
> From: Simon St.Laurent [mailto:simonstl@simonstl.com]
> Sent: Thursday, September 03, 2009 11:54 AM
> To: xml-dev@lists.xml.org
> Subject: [xml-dev] XML not ideal for Big Data
> Perhaps there were better ways to have made XML work with his
> problems... but I think on the whole he's right.
> http://dataspora.com/blog/xml-and-big-data/
> --
> Simon St.Laurent
> http://simonstl.com/
> ____________________________


XML-DEV is a publicly archived, unmoderated list hosted by OASIS
to support XML implementation and development. To minimize
spam in the archives, you must subscribe before posting.

[Un]Subscribe/change address: http://www.oasis-open.org/mlmanage/
Or unsubscribe: xml-dev-unsubscribe@lists.xml.org
subscribe: xml-dev-subscribe@lists.xml.org
List archive: http://lists.xml.org/archives/xml-dev/
List Guidelines: http://www.oasis-open.org/maillists/guidelines.php

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]

News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 1993-2007 XML.org. This site is hosted by OASIS