OASIS Mailing List ArchivesView the OASIS mailing list archive below
or browse/search using MarkMail.


Help: OASIS Mailing Lists Help | MarkMail Help

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: [xml-dev] Whitespace handling when compressing XML with WInZIP

I should add, I did the same test with gzip and get roughly the same
Hence my enquiry as to a gzip "dump" program.


                    Eric Bohlman                                                                                                         
                    <ebohlman@eart        To:     xml-dev@lists.xml.org                                                                  
                    hlink.net>            cc:                                                                                            
                                          Subject:     Re: [xml-dev] Whitespace handling when compressing XML with WInZIP                
                    03:13 AM                                                                                                             

9/28/01 10:15:39 PM, Mike_Leditschke@nemmco.com.au wrote:

>I have an XML file with most of the data at a tag nesting level of 5-7,
>and am compressing it with WINZip 8.
>While not touching the markup, I've found the whitespace used
>to "step out" the tags has a big impact on the compression ratio
>Using 4 chars per nesting level, the file is 6.7 MB and compresses
>to 154Kb.
>Using  1 tab per nesting level, the file is 4.2 MB and compresses
>to 98 Kb.
>While the change in the input file is what I'd anticipated, I would have
>expected the zip file to be roughly the same size. Wouldn't
>repeated runs of the same character  be replaced with a single token,
>whether they be spaces or tabs?
>Am I missing something obvious?  Does anyone understand the
>internals of WINZIP enough to explain the descrepancy? The ratio
>of the compressed files suggests its encoding the multiple spaces
>with multiple tokens rather than a single token.

If your data contains repeated occurrences of, say


all but the first such occurrence will be replaced with a single token.
But if it contains




and the like, it will need a different token for each of them and things
will start to pile up.
Remember that the compression algorithm doesn't just squash repeated runs
of single characters; it
tries to write "shorthand" for entire "phrases" that repeat in the
document.  The longer the
repeated "phrases," the better the compression.  Lots of "phrases" that are
almost identical
except in the middle don't compress as well, and that's what varying levels
of whitespace give

The xml-dev list is sponsored by XML.org <http://www.xml.org>, an
initiative of OASIS <http://www.oasis-open.org>

The list archives are at http://lists.xml.org/archives/xml-dev/

To subscribe or unsubscribe from this elist use the subscription
manager: <http://lists.xml.org/ob/adm.pl>

This e-mail is confidential.  If you are not the intended recipient, any use, disclosure or copying of this document is unauthorised and prohibited.  If you have received this document in error, please delete the email and notify me by return email or by phoning the NEMMCO Helpdesk on 1300 300 295.