view release on metacpan or  search on metacpan

unzip-6.0/proginfo/ziplimit.txt  view on Meta::CPAN

    Zip archive processing requires random access to the archive file for
    jumping between different parts of the archive's structure.
    In standard C, this is done via stdio functions fseek()/ftell() resp.
    unix-io functions lseek()/tell().  In many (most?) C implementations,
    these functions use "signed long" variables to hold offset pointers
    into sequential files.  In most cases, this is a signed 32-bit number,
    which is limited to ca. 2E+09.  There may be specific C runtime library
    implementations that interpret the offset numbers as unsigned, but for
    us, this is not reliable in the context of portable programming.

 2. Similarly, for executables without "Zip64" and "LargeFile" support,
    the 2GiByte limit on the size of a single compressed archive member
    is again a consequence of the implementation in C.
    The variables used internally to count the size of the compressed
    data stream are of type "long", which is guaranted to be at least
    32-bit wide on all supported environments.

    But, why do we use "signed" long and not "unsigned long"?

    Throughout the I/O handling of the compressed data stream, the sign bit
    of the "long" numbers is (mis-)used as a kind of overflow detection.
    In the end, this is caused by the fact that standard C lacks any
    overflow checking on integer arithmetics and does not support access
    to the underlying hardware's overflow detection (the status bits,
    especially "carry" and "overflow" of the CPU's flags-register) in a
    system-independent manner.

    So, we "misuse" the most-significant bit of the compressed data size
    counters as carry bit for efficient overflow/underflow detection.  We
    could change the code to a different method of overflow detection, by
    using a bunch of "sanity" comparisons (kind of "is the calculated result
    plausible when compared with the operands"). But, this would "blow up"
    the code of the "inner loop", with remarkable loss of processing speed.
    Or, we could reduce the amount of consistency checks of the compressed
    data (e.g. detection of premature end of stream) to an absolute minimum,
    at the cost of the programs' stability when processing corrupted data.

 3. The argumentation above is somewhat out-dated. Beginning with the
    releases of Zip 3 and UnZip 6, Info-ZIP programs support archive
    sizes larger than 4GiB on systems where the required underlying
    support for 64-bit file offsets and file sizes is available from
    the OS (and the C runtime environment).

    For executables with support for "Zip64" archive format and "LargeFile"
    extension, the I/O limits are lifted by applying extended 64-bit off_t
    file offsets.  All limits discussed above are then based on integer
    sizes of 64 bits instead of 32, this should allow to handle file and
    archive sizes up to the limits of manufacturable hardware for the
    foreseeable future.  The reduction of the theoretical limits from
    (2^64 - 1) to (2^63 - 1) because of the throughout use of signed
    numbers can be neglected with the currently imaginable hardware.

    However, this new support partially breaks compatibility with older
    "legacy" systems.  And it should be noted that the portability and
    readability of the UnZip and Zip code has suffered somehow caused
    by the extensive use of non-standard language extension needed for
    64-bit support on the major target systems.

Please report any problems to:  Zip-Bugs at www.info-zip.org

Last updated:  25 May 2008, Ed Gordon
               02 January 2009, Christian Spieler