view release on metacpan or  search on metacpan

Dynagzip.pm  view on Meta::CPAN

       PerlSetVar BinaryCGI On
       PerlSetVar LightCompression On
       Order allow,deny
       Allow from all
   </Directory>

 client-side log:

  C05 --> S06 GET /cgi-bin/mylook.cgi HTTP/1.1
  C05 --> S06 Accept: */*
  C05 --> S06 Accept-Language: en-us
  C05 --> S06 Accept-Encoding: gzip, deflate
  C05 --> S06 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
  C05 --> S06 Host: devl4.outlook.net
  C05 --> S06 Pragma: no-cache
  C05 --> S06 Accept-Charset: ISO-8859-1
  == Body was 0 bytes ==

  C05 <-- S06 HTTP/1.1 200 OK
  C05 <-- S06 Date: Fri, 31 May 2002 23:37:45 GMT
  C05 <-- S06 Server: Apache/1.3.22 (Unix) Debian GNU/Linux mod_perl/1.26
  C05 <-- S06 X-Module-Sender: Apache::Dynagzip
  C05 <-- S06 Transfer-Encoding: chunked
  C05 <-- S06 Expires: Friday, 31-May-2002 23:42:45 GMT
  C05 <-- S06 Vary: Accept-Encoding
  C05 <-- S06 Content-Type: text/html; charset=iso-8859-1
  C05 <-- S06 Content-Encoding: gzip
  C05 <-- S06 == Incoming Body was 994 bytes ==
  == Transmission: text gzip chunked ==
  == Chunk Log ==
  a (hex) = 10 (dec)
  3c7 (hex) = 967 (dec)
  0 (hex) = 0 (dec)
  == Latency = 0.170 seconds, Extra Delay = 0.110 seconds
  == Restored Body was 1862 bytes ==

=head1 INTRODUCTION

From a historical point of view this package was developed primarily in order to compress the output
of a proprietary CGI binary written in C that was
widely used by Outlook Technologies, Inc. in order to deliver uncompressed dynamically generated
HTML content over the Internet using C<HTTP/1.0> since the mid-'90s.
We were then presented with the challenge of using the content compression
features over C<HTTP/1.1> on busy production servers, especially those serving heavy traffic on virtual hosts
of popular American broadcasting companies.

The very first our attempts to implement a static gzip approach in order to compress the
dynamic content helped us to scale effectively the bandwidth at
the cost of significantly increased latency of the content delivery.

That was why I came up with an idea to use chunked data transmission of
the gzipped content, sharing a real time between the server side data
creation/compression, media data transmission, and the client side data
decompression/presentation in order to provide end users with the partially
displayed content as soon as it's possible in particular conditions of the
user's connection.

At the time we decided to go for dynamic compression there were no
appropriate software on the market. Even later in
February 2002 Nicholas Oxhøj wrote to the mod_perl mailing list about his
experience of finding Apache gzipper for the streaming outgoing content:

=for html
<blockquote>

I<"... I have been experimenting with all the different Apache compression modules
I have been able to find, but have not been able to get the desired result.
I have tried Apache::GzipChain, Apache::Compress, mod_gzip and mod_deflate, with
different results.  One I cannot get to work at all. Most work, but seem to collect
all the output before compressing it and sending it to the browser...>

I<... Wouldn't it be nice to have some option to specify that the handler should flush
and send the currently compressed output every time it had received a certain amount
of input or every time it had generated a certain amount of output?..>

I<... So I am basically looking for anyone who has had any success in achieving this
kind of "streaming" compression, who could direct me at an appropriate Apache module.">

=for html
</blockquote>

Unfortunately for him, C<Apache::Dynagzip> has not yet been publicly available at that time...

Since relesed this handler is especially useful when one needs to compress the outgoing
web content that is dynamically generated on the fly using templates,
DB data, XML, etc., and when at the time of the request it is impossible
to determine the length of the response.

Content provider can benefit additionally from the fact that handler begins the transmission
of compressed data concurent to further document creation.
On the other hand, the internal buffer inside the
handler prevents Apache from the creation of too short chunks over C<HTTP/1.1>.

In order to simplify the use of this handler on public/open-source sites,
the capability of content compression over HTTP/1.0 was added to this handler since the version 0.06.
This helps to avoid dynamic invocation of other Apache handlers
for the content generation phase.

=head2 Acknowledgments

Thanks to Tom Evans, Valerio Paolini, and Serge Bizyayev for their valuable idea contributions and multiple testing.
Thanks to Igor Sysoev and Henrik Nordstrom those helped me to understand better the HTTP/1.0 compression features.
Thanks to Vlad Jebelev for the patch that helps to survive possible dynamical Apache downgrade
from HTTP/1.1 to HTTP/1.0 (especially serving MSIE requests over SSL).
Thanks to Rob Bloodgood and Damyan Ivanov for the patches those help to eliminate some unnecessary warnings.
Thanks to John Siracusa for the hint that helps to control the content type properly.
Thanks to Richard Chen for the bug report concerning some uncompressed responses.

Obviously, I hold a full responsibility for how all those contributions are implemented.

=head1 DESCRIPTION

The main pupose of this package is to serve the C<content generation phase> within the mod_perl enabled
C<Apache 1.3.X>, providing dynamic on the fly compression of outgoing web content.
This is done through the use of C<zlib> library via the C<Compress::Zlib> perl interface
to serve both C<HTTP/1.0> and C<HTTP/1.1> requests from clients/browsers,
capable to understand C<gzip> format and decompress it on the fly.
This handler does never C<gzip> content for clients/browsers
those do not declare the ability to decompress C<gzip> format.

In fact, this handler mainly serves as a kind of
customizable filter of outbound web content for C<Apache 1.3.X>.

This handler is supposed to be used within C<Apache::Filter> chain mostly in order to serve the
outgoing content that is dynamically generated on the fly by Perl and/or Java.
It is featured to serve the regular CGI binaries (C-written for examle)
as a standalong handler out of C<Apache::Filter> chain.
As an extra option, this handler can be used to compress dynamically the huge static
files, and to transfer gzipped content in the form of a stream back to the
client browser. For the last purpose C<Apache::Dynagzip> handler should be configured as
a standalong handler out of C<Apache::Filter> chain too.

Working over C<HTTP/1.0> this handler indicates the end of data stream by closing connection.
Indeed, over C<HTTP/1.1> the outgoing data is compressed within a chunked outgoing stream,
keeping the connection alive. Resonable control over the chunk-size is provided in this case.

In order to serve better the older web clients,