Re: [xml-dev] Whitespace handling when compressing XML with WInZIP

[Mike_Leditschke@nemmco.com.au]

I should add, I did the same test with gzip and get roughly the same
numbers.
Hence my enquiry as to a gzip "dump" program.

Regards
Michael



                                                                                                                                         
                    Eric Bohlman                                                                                                         
                    <ebohlman@eart        To:     xml-dev@lists.xml.org                                                                  
                    hlink.net>            cc:                                                                                            
                                          Subject:     Re: [xml-dev] Whitespace handling when compressing XML with WInZIP                
                    30/09/2001                                                                                                           
                    03:13 AM                                                                                                             
                                                                                                                                         
                                                                                                                                         




9/28/01 10:15:39 PM, Mike_Leditschke@nemmco.com.au wrote:

>I have an XML file with most of the data at a tag nesting level of 5-7,
>and am compressing it with WINZip 8.
>
>While not touching the markup, I've found the whitespace used
>to "step out" the tags has a big impact on the compression ratio
>achievable.
>
>
>Using 4 chars per nesting level, the file is 6.7 MB and compresses
>to 154Kb.
>
>Using  1 tab per nesting level, the file is 4.2 MB and compresses
>to 98 Kb.
>
>While the change in the input file is what I'd anticipated, I would have
>expected the zip file to be roughly the same size. Wouldn't
>repeated runs of the same character  be replaced with a single token,
>whether they be spaces or tabs?
>
>Am I missing something obvious?  Does anyone understand the
>internals of WINZIP enough to explain the descrepancy? The ratio
>of the compressed files suggests its encoding the multiple spaces
>with multiple tokens rather than a single token.

If your data contains repeated occurrences of, say

</end>
<start>

all but the first such occurrence will be replaced with a single token.
But if it contains

</end>
<start>

  </end>
  <start>

    </end>
  <start>

and the like, it will need a different token for each of them and things
will start to pile up.
Remember that the compression algorithm doesn't just squash repeated runs
of single characters; it
tries to write "shorthand" for entire "phrases" that repeat in the
document.  The longer the
repeated "phrases," the better the compression.  Lots of "phrases" that are
almost identical
except in the middle don't compress as well, and that's what varying levels
of whitespace give
you.



-----------------------------------------------------------------
The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
initiative of OASIS <http://www.oasis-open.org>

The list archives are at http://lists.xml.org/archives/xml-dev/

To subscribe or unsubscribe from this elist use the subscription
manager: <http://lists.xml.org/ob/adm.pl>






------------------------------------------
This e-mail is confidential.  If you are not the intended recipient, any use, disclosure or copying of this document is unauthorised and prohibited.  If you have received this document in error, please delete the email and notify me by return email or by phoning the NEMMCO Helpdesk on 1300 300 295.