[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [xml-dev] Whitespace handling when compressing XML with WInZIP
- From: Eric Bohlman <firstname.lastname@example.org>
- To: email@example.com
- Date: Sat, 29 Sep 2001 12:13:03 -0500
9/28/01 10:15:39 PM, Mike_Leditschke@nemmco.com.au wrote:
>I have an XML file with most of the data at a tag nesting level of 5-7,
>and am compressing it with WINZip 8.
>While not touching the markup, I've found the whitespace used
>to "step out" the tags has a big impact on the compression ratio
>Using 4 chars per nesting level, the file is 6.7 MB and compresses
>Using 1 tab per nesting level, the file is 4.2 MB and compresses
>to 98 Kb.
>While the change in the input file is what I'd anticipated, I would have
>expected the zip file to be roughly the same size. Wouldn't
>repeated runs of the same character be replaced with a single token,
>whether they be spaces or tabs?
>Am I missing something obvious? Does anyone understand the
>internals of WINZIP enough to explain the descrepancy? The ratio
>of the compressed files suggests its encoding the multiple spaces
>with multiple tokens rather than a single token.
If your data contains repeated occurrences of, say
all but the first such occurrence will be replaced with a single token. But if it contains
and the like, it will need a different token for each of them and things will start to pile up.
Remember that the compression algorithm doesn't just squash repeated runs of single characters; it
tries to write "shorthand" for entire "phrases" that repeat in the document. The longer the
repeated "phrases," the better the compression. Lots of "phrases" that are almost identical
except in the middle don't compress as well, and that's what varying levels of whitespace give