OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

 


 

   Re: [xml-dev] To improve XML file creation and parsing performance

[ Lists Home | Date Index | Thread Index ]

Does the software inserting records into the new database perform batch 
inserts? Or does it insert one record at a time? This can greatly affect 
performance due to the number of INSERT statements being performed.

Note that writing software to perform batch inserts is difficult if that 
software is designed to handle deeply nested XML [1] as opposed to 
tabular XML [2]. The reason is that in deeply nested XML, the software 
must cache records of a similar type until it has enough records to 
perform a batch insert. If the XML is tabular, the software can simply 
insert records as it processes the file.

Therefore, if you're writing your own software, you might want to use 
tabular XML and perform batch inserts. If you're using somebody else's 
software and it handles deeply nested XML, chances are that it doesn't 
do batch inserts.

Another solution here is to use XSLT or custom software (including a 
custom, schema-specific parser) to convert the data to the bulk load 
syntax used by the database and use the bulk loading facilities of the 
database. I've heard of at least one company that experimented with this 
and got better performance.

-- Ron

[1] Deeply nested documents impose a hierarchy over the graph formed by 
the relational schema. For example, line item elements are nested inside 
their corresponding sales order elements.

[2] Tabular documents have the hierarchical form database / tables / 
table / row / column, using the obvious mapping from database to XML. 
For example, line item elements and sales order elements are stored in 
parallel at the same level of nesting.

Chiusano Joseph wrote:
> How large is your XML file? Also, the issue may not be with the XML file
> itself (or at least not all of it) - it may have to do with the target
> database (i.e. is it properly tuned, are the proper indexes present and
> active, etc.)
> 
> Kind Regards,
> Joseph Chiusano
> Booz Allen Hamilton
> O: 703-902-6923
> C: 202-251-0731
> Visit us online@ http://www.boozallen.com
>  
> 
> 
>>-----Original Message-----
>>From: Sube Singh [mailto:sube.verma@gmail.com] 
>>Sent: Tuesday, August 16, 2005 2:55 AM
>>To: xml-dev@lists.xml.org; alan-xml-dev@engrm.com
>>Subject: [xml-dev] To improve XML file creation and parsing 
>>performance
>>
>>Hi,
>>
>>We would like to create a XML file from a database. After 
>>creation of XML file we will write it on client socket. After 
>>getting file at client end we will parse the XML file and 
>>insert datas in client database.There are 10 Lakhs records in 
>>database. It do this work XML take update 3 hours.
>>
>>Can any one suggest us how to improve the XML file parsing 
>>and creation performance.
>>
>>We hope that will get the solution of problem with help the 
>>help of your valueable suggestion.
>>
>>Regards,
>>
>>-Sube





 

News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS