OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.

 


Help: OASIS Mailing Lists Help | MarkMail Help

 


 

   Re: [xml-dev] Out of memory error with XML and Java ResultSet

[ Lists Home | Date Index | Thread Index ]

Hi Jason,

Do you know the typical size of the XML text generated from each row? 
Also, do you need to have the full document buffered in memory or do you 
really want it as a stream that's being sent somewhere else?

In general, you're going to want to avoid any approach (such as DOM) 
that builds a representation of the XML structure in memory. Even just 
buffering the output text in memory is probably going to cause problems 
if this gets longer than a few MB. Your best bet is probably just to 
stream it out, either directly to the destination or to a file on the 
local system.

  - Dennis

Brown, Jason B. wrote:

>Greetings.
>
>I am new to this mailing list.  I was wondering if someone could help me
>with a small problem I am having trying to create an XML buffer using
>data from a Java ResultSet.  The ResultSet object contains approximately
>300,000 rows, which I want to place in a XML buffer.  I managed to
>develop an XML implementation that uses JAXP 1.1.  The logic for the
>processing is this: I execute the SQL query, obtain the ResultSet, loop
>through the result set and as each row in the result set is encountered,
>I use getLong methods, etc to get the data and write it to a
>StreamResult object.
>
>Here is a code snippet:
>			rsMetaData = rs.getMetaData();
>
>			// Provide the System.out object as the output
>stream for now
>			xmlWrtr = new CisXmlWriterSAX(new
>StreamResult(xmlData), 
>							((String)
>CUtil.getEnv("AdhocDataDtd")));
>
>			numOfColumns = rsMetaData.getColumnCount();
>
>			//
>~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
>			// Start XML document
>			xmlWrtr.startXMLDocument();
>
>			// Set root element
>			xmlWrtr.addXMLElement("AdhocClaims");
>
>			System.out.println("<<>> Building XML file...");
>
>			while (rs.next()) {
>				// Calculate number of rows in result
>set
>				numOfRows++;
>				
>				CDebug.print("" + numOfRows);
>
>				// Create the Row parent element (child
>element to AdhocClaims element)
>				xmlWrtr.addXMLElement("Row");
>
>				for (int colIndex = 1; colIndex <=
>numOfColumns; colIndex++) {
>
>				.....
>					tempStr =
>crossRef.getReadableName(rsMetaData.getColumnName(colIndex));
>
>					// Create the attributes for the
>element
>					xmlWrtr.addElementAttr("name",
>"CDATA", tempStr);
>
>					// Add Column element
>					xmlWrtr.addXMLElement("Column");
>
>					// Add the Value element and a
>value for the Value element
>					xmlWrtr.addElementValue(null,
>"");
>
>					// End Column element
>					xmlWrtr.endXMLElement("Column");
>				}
>				.....
>			}
>
>			xmlWrtr.endXMLElement("AdhocClaims");
>			
>			// This is done when finished with adding all
>elements within root element
>			xmlWrtr.endXMLDocument();
>			......
>
>xmlWrtr is a class that contains the following objects:
>	private SAXTransformerFactory transFact = null;
>
>	/**
>	* Contains where the resulting XML will be placed/contained
>	*/
>	private StreamResult resultStream = null;
>	
>	private TransformerHandler handler = null;
>
>	private Transformer serializer = null;
>
>	private AttributesImpl attr = null;
>
>My main goal with this posting is to obtain a memory-friendly solution
>for taking information from a Java ResultSet and placing it in a XML
>buffer.
>
>Thanks.
>
>Jason B. Brown
>CIS IS Claims
>215-241-4609
>x2-4609 (internal), 11th Floor
>mailto:jason.b.brown@ibx.com
>
>
>
>CONFIDENTIALITY NOTICE: This E-Mail is intended only 
>for the use of the individual or entity to which it is addressed and may contain information that is privileged, confidential and exempt from disclosure under applicable law. If you have received this communication in error, please do not distribute and delete the original message.  Please notify the sender by E-Mail at the address shown. Thank you for your compliance..
>
>
>-----------------------------------------------------------------
>The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
>initiative of OASIS <http://www.oasis-open.org>
>
>The list archives are at http://lists.xml.org/archives/xml-dev/
>
>To subscribe or unsubscribe from this list use the subscription
>manager: <http://lists.xml.org/ob/adm.pl>
>
>  
>





 

News | XML in Industry | Calendar | XML Registry
Marketplace | Resources | MyXML.org | Sponsors | Privacy Statement

Copyright 2001 XML.org. This site is hosted by OASIS