OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]
Re: [xml-dev] What is the general direction you are seeing thesedays to store and query lots of large complex =?UTF-8?Q?XML=3F?=

On 2015-03-05 19:05, Gareth Oakes wrote:
Querying 2.4PB (2400TB) of data is going to take a
long time regardless of how you do it.
The best way is probably not doing it, where possible - e.g. project a subset and query that.

MarkLogic is marketed as petabyte-scale,
but XML may not be the most optimal representation of the data, it is
worth considering the alternatives.
Luckily neither MarkLogic nor most other XML-native databases store XML; rather, they store an internal representation that can be mapped to the XPath and XQuery Data Model (the XDM).

A common mistake people make (I'm not saying you're making it, but people reading your post might jump to an incorrect conclusion) is thinking that XML databases store XML (e.g. store pointy brackets), which is a bit like thinking that relational databases operate on CSV files.

Distributed storage and query are probably the way forward,
Depending on the nature of the queries, that does seem likely.

Liam Quin, W3C
XML Activity Lead;
Digital publishing; HTML Accessibility

[Date Prev] | [Thread Prev] | [Thread Next] | [Date Next] -- [Date Index] | [Thread Index]

News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 1993-2007 XML.org. This site is hosted by OASIS