[
Lists Home |
Date Index |
Thread Index
]
Brown, Jason B. wrote:
>One row of XML text is approximately 230 characters, including tag name
>and attribute specifiers.
>
>
So you have about 230 characters x 300,000 rows. 70,000,000 characters.
In Java, a character
takes two bytes, so that is 140,000,000 bytes. If you are appending to a
StringBuffer, it
doubles the size of the underlying array whenever it needs more; at
worse case, if your
StringBuffer had a full array with 70,000,000 characters and you
appended one more character,
then StringBuffer would allocate a new underlying array of 140,000,000
characters. So that
takes up 280,000,000 bytes. *Plus* not forgetting that the previous
array was still needed at the
same time as the new one, so at the time of expanding the capacity of
the underlying array,
you needed 420,000,000 bytes!
And that apart from anything else going on in your program.
I suggest you:--
1) Allocate the StringBuffer with the correct amount of size (e.g. 70
million characters)
to prevent the worst-case blowout and reallocation requirements; If you
regularly have
to handle large StringBuffers, you should add it to your QA checklist to
check that
all large StringBuffers that are appended to should have preallocated
capacities on
initialization;
2) Make sure your JVM has been allocated with enough heap (-Xms);
3) Double check your code so that you make sure the StringBuffer is
explicitly set to null
as soon as it is not needed (which may help garbage collection), and
that any external
references to the StringBuffer free it as soon as needed.
Cheers
Rick Jelliffe
|
