[
Lists Home |
Date Index |
Thread Index
]
Does the software inserting records into the new database perform batch
inserts? Or does it insert one record at a time? This can greatly affect
performance due to the number of INSERT statements being performed.
Note that writing software to perform batch inserts is difficult if that
software is designed to handle deeply nested XML [1] as opposed to
tabular XML [2]. The reason is that in deeply nested XML, the software
must cache records of a similar type until it has enough records to
perform a batch insert. If the XML is tabular, the software can simply
insert records as it processes the file.
Therefore, if you're writing your own software, you might want to use
tabular XML and perform batch inserts. If you're using somebody else's
software and it handles deeply nested XML, chances are that it doesn't
do batch inserts.
Another solution here is to use XSLT or custom software (including a
custom, schema-specific parser) to convert the data to the bulk load
syntax used by the database and use the bulk loading facilities of the
database. I've heard of at least one company that experimented with this
and got better performance.
-- Ron
[1] Deeply nested documents impose a hierarchy over the graph formed by
the relational schema. For example, line item elements are nested inside
their corresponding sales order elements.
[2] Tabular documents have the hierarchical form database / tables /
table / row / column, using the obvious mapping from database to XML.
For example, line item elements and sales order elements are stored in
parallel at the same level of nesting.
Chiusano Joseph wrote:
> How large is your XML file? Also, the issue may not be with the XML file
> itself (or at least not all of it) - it may have to do with the target
> database (i.e. is it properly tuned, are the proper indexes present and
> active, etc.)
>
> Kind Regards,
> Joseph Chiusano
> Booz Allen Hamilton
> O: 703-902-6923
> C: 202-251-0731
> Visit us online@ http://www.boozallen.com
>
>
>
>>-----Original Message-----
>>From: Sube Singh [mailto:sube.verma@gmail.com]
>>Sent: Tuesday, August 16, 2005 2:55 AM
>>To: xml-dev@lists.xml.org; alan-xml-dev@engrm.com
>>Subject: [xml-dev] To improve XML file creation and parsing
>>performance
>>
>>Hi,
>>
>>We would like to create a XML file from a database. After
>>creation of XML file we will write it on client socket. After
>>getting file at client end we will parse the XML file and
>>insert datas in client database.There are 10 Lakhs records in
>>database. It do this work XML take update 3 hours.
>>
>>Can any one suggest us how to improve the XML file parsing
>>and creation performance.
>>
>>We hope that will get the solution of problem with help the
>>help of your valueable suggestion.
>>
>>Regards,
>>
>>-Sube
|
