RE: [xml-dev] storing XML files

[Chris Parkerson <chrisp@exceloncorp.com>]

Albena:

Whether you need a native XML database or whether the XML capabilities
of the RDBMS systems will be sufficient is all dependent upon the type
of data you're dealing with and the application.

If the documents are going to be sufficiently small (less than a few
hundred KB of raw XML each), or can be quickly partitioned using XSLT to
be sufficiently small, their schemas do not change often (i.e. the feeds
you are receiving hold pretty much to a fixed schema), and being able to
do fast queries across large document sets is NOT a requirement (i.e.
you've got 1000+ stock quote documents, but you do not need to query
over them as an aggregate), then the XML capabilities of the RDBMS
vendors should be sufficient.  Microsoft's SQL Server is the furthest
ahead in terms of XML support and is the only RDBMS vendor at present to
support incremental updates to XML documents via update grams.  Oracle
9i's XML support is really limited with their new vaunted "XMLType"
column data type being nothing more than a convenience wrapper around
the CLOB type: XML still gets stored as a blob.  Keep in mind, however,
that each RDBMS vendors approach to handling XML is going to be
proprietary: you will have little to no code or knowledge portability.

A major advantage of native XML DB systems (save for Software AG's
Tamino which still uses a proprietary query language and schema dialect,
among other non-standard things) is their adherence to standards.
Queries are XPath, transformations are XSLT, granular access to document
data is via the standard DOM API, validation can be against DTD or W3C
XML Schema, etc.  You gain a bit more in terms of code portability as
well as knowledge portability between different vendors.  You also gain
a database system inherently designed to deal with the extensibility of
XML data and capable of removing the scalability, performance, ease of
data update, and cross-document limitations of the RDBMS approach.

I would recommend you give native XML databases a try (eXcelon's
Extensible Information Server is available for free 30-day download from
our homepage at www.exceloncorp.com and is an enterprise-class, fully
transactional native XML database... there are others out there, of
course, with a good description of each found at
http://www.rpbourret.com/xml/XMLDatabaseProds.htm), but also thoroughly
investigate the capabilities of the RDBMS vendors, especially Microsoft
SQL Server.  The reason most XML applications are converting to native
XML databases is that they've tried to make their applications work with
RDBMS systems and they failed (your application example here is very
similar to a problem faced by one of our own customers, SwissRe
Insurance... they started with Oracle, ended up with eXcelon).  If you
get yours to work well with an RDBMS (and of course, there are those
that work just fine), then you really don't need a native XML DB.

Hope this helps,
Chris

---------------------------------------
Chris Parkerson
Product Manager - XML Platform
eXcelon Corporation
Burlington, MA
(781) 674-5393
http://www.exceloncorp.com
---------------------------------------


-----Original Message-----
From: Albena Georgieva [mailto:albena@info.nl] 
Sent: Monday, October 08, 2001 11:38 AM
To: 'xml-dev@lists.xml.org'
Subject: [xml-dev] storing XML files

Dear All,

I am given to make a decision for which I don't have enough experience
and
knowledge, so I am addressing you for an advice.

The general idea is storing several different data feeds, which arrive
in
XML format into a database. The data feeds can be very different (stock
quotes, weather report, events information, daily news, horoscope, ...)
and
we receive them in XML format from different sources.
Further, they are displayed on the WAP site, according to the search
criteria of the user. Example: (select all the movies today in Amsterdam
or
show me the horoscope for THIS sign).
After certain time the data feeds expire. 


Ideally, I would like to transform all of those different XMLs into some
general
format, from which sql output can be generated.
I have though of SQL because the current database of use by the WAP site
is
SQL Server. The information is accessed through ASP files and store
procedures.
There is a slight possibility that the customer moves to Oracle, so my
solution shouldn't be database provider dependent.

I have many, many questions, but the most frightening once are:
Is it smart to develop a generalized format into which all those data
feeds
will be transformed (I hope an XSLT is a good tool to do that)? Or, I
should
deal with them individually?
Can I apply XSLT on my XML data feeds and directly produce SQL
statements?
Has someone already developed an XML2SQL transformation tool? Just a
flat
table solution or I can also capture relationship information?
Is it the right thing to save those XMLs in to a relational database or
shall I explore the native XML databases?
Is there an existing DTD for SQL or that makes no sense to you?
My system should be flexible enough to accept new kinds of data feeds
...

Hope, I made my worries, doubts and knowledge holes clear. 
Any suggestion is welcome.

Thanx,
Albena





-----------------------------------------------------------------
The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
initiative of OASIS <http://www.oasis-open.org>

The list archives are at http://lists.xml.org/archives/xml-dev/

To subscribe or unsubscribe from this elist use the subscription
manager: <http://lists.xml.org/ob/adm.pl>