[
Lists Home |
Date Index |
Thread Index
]
Last time someone told me they had a big XML file it turned out to be only
500Kb, so my advice was all wrong. What do you mean by "big"?
Michael Kay
http://www.saxonica.com/
> -----Original Message-----
> From: william.j.boxall@accenture.com
> [mailto:william.j.boxall@accenture.com]
> Sent: 07 January 2005 17:01
> To: xml-dev@lists.xml.org
> Subject: [xml-dev] Large xml processing into SQL Server
>
> Hello, all.. I just joined this list and have a question that
> may have been answered by someone before, but here it is.
>
> At work, we're working on a BEA Weblogic project (Java), and
> part of it is I wrote an interface that takes an xml file,
> processes it with a validation routine, and then stores it in
> a database. I have found, though, that when they send me a
> large xml file, I can't handle it. I get a transaction
> timeout, which is set by Weblogic, but that can be increased.
> But then I run into out of memory issues. I've done a lot
> of research, and the best solution appears to be to figure
> out a way to bulk load the xml document into SQL Server and
> then do my data manipulation. For some reason, even when I
> trim down the operations so that all I'm doing is a straight
> insert into the database, it still takes way too long and times out.
>
> I am not an xml guru at all, and am getting a bit of an
> education in the process of trying to find a workable
> solution. Is there a bulk loader I can use? If so, where
> and how do I use it? Should I use an embedded xml database
> for processing? If so, are there detailed instructions, and
> will that alleviate in any way storing the xml records into
> SQL Server?
>
> I cannot load the entire xml file into memory, it's too big.
> Is there some sort of streaming operation I can take advantage of?
>
> One solution I was messing with was to take the schema and
> map it to the database fields through using sql:relation,
> etc.. But the problem there is the schema is built from the
> xml file, which is hierarchical in nature. I need to flatten
> out the elements in the hierarchy in order to do the mapping
> to database fields. I've searched all over the net for a way
> to do this, and have found some cryptic solutions using xslt,
> which I am totally unfamiliar with. Again, any help with
> that would be gratefully appreciated.
>
> Thank you in advance!
> Bill
>
>
> This message is for the designated recipient only and may
> contain privileged, proprietary, or otherwise private
> information. If you have received it in error, please notify
> the sender immediately and delete the original. Any other
> use of the email by you is prohibited.
>
> -----------------------------------------------------------------
> The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
> initiative of OASIS <http://www.oasis-open.org>
>
> The list archives are at http://lists.xml.org/archives/xml-dev/
>
> To subscribe or unsubscribe from this list use the subscription
> manager: <http://www.oasis-open.org/mlmanage/index.php>
>
>
|
