view release on metacpan or  search on metacpan

Dynagzip.pm  view on Meta::CPAN

  3cf (hex) = 975 (dec)
  0 (hex) = 0 (dec)
  == Latency = 0.110 seconds, Extra Delay = 0.110 seconds
  == Restored Body was 1954 bytes ==

 =================================================================================
 GET dynamically generated (by C-written binary) html file with light compression:
 =================================================================================
  httpd.conf:

   PerlModule Apache::Dynagzip
   <Directory /var/www/cgi-bin/>
       SetHandler perl-script
       PerlHandler Apache::Dynagzip
       AllowOverride None
       Options +ExecCGI
       PerlSetupEnv On
       PerlSetVar BinaryCGI On
       PerlSetVar LightCompression On
       Order allow,deny
       Allow from all
   </Directory>

 client-side log:

  C05 --> S06 GET /cgi-bin/mylook.cgi HTTP/1.1
  C05 --> S06 Accept: */*
  C05 --> S06 Accept-Language: en-us
  C05 --> S06 Accept-Encoding: gzip, deflate
  C05 --> S06 User-Agent: Mozilla/4.0 (compatible; MSIE 6.0; Windows 98)
  C05 --> S06 Host: devl4.outlook.net
  C05 --> S06 Pragma: no-cache
  C05 --> S06 Accept-Charset: ISO-8859-1
  == Body was 0 bytes ==

  C05 <-- S06 HTTP/1.1 200 OK
  C05 <-- S06 Date: Fri, 31 May 2002 23:37:45 GMT
  C05 <-- S06 Server: Apache/1.3.22 (Unix) Debian GNU/Linux mod_perl/1.26
  C05 <-- S06 X-Module-Sender: Apache::Dynagzip
  C05 <-- S06 Transfer-Encoding: chunked
  C05 <-- S06 Expires: Friday, 31-May-2002 23:42:45 GMT
  C05 <-- S06 Vary: Accept-Encoding
  C05 <-- S06 Content-Type: text/html; charset=iso-8859-1
  C05 <-- S06 Content-Encoding: gzip
  C05 <-- S06 == Incoming Body was 994 bytes ==
  == Transmission: text gzip chunked ==
  == Chunk Log ==
  a (hex) = 10 (dec)
  3c7 (hex) = 967 (dec)
  0 (hex) = 0 (dec)
  == Latency = 0.170 seconds, Extra Delay = 0.110 seconds
  == Restored Body was 1862 bytes ==

=head1 INTRODUCTION

From a historical point of view this package was developed primarily in order to compress the output
of a proprietary CGI binary written in C that was
widely used by Outlook Technologies, Inc. in order to deliver uncompressed dynamically generated
HTML content over the Internet using C<HTTP/1.0> since the mid-'90s.
We were then presented with the challenge of using the content compression
features over C<HTTP/1.1> on busy production servers, especially those serving heavy traffic on virtual hosts
of popular American broadcasting companies.

The very first our attempts to implement a static gzip approach in order to compress the
dynamic content helped us to scale effectively the bandwidth at
the cost of significantly increased latency of the content delivery.

That was why I came up with an idea to use chunked data transmission of
the gzipped content, sharing a real time between the server side data
creation/compression, media data transmission, and the client side data
decompression/presentation in order to provide end users with the partially
displayed content as soon as it's possible in particular conditions of the
user's connection.

At the time we decided to go for dynamic compression there were no
appropriate software on the market. Even later in
February 2002 Nicholas Oxhøj wrote to the mod_perl mailing list about his
experience of finding Apache gzipper for the streaming outgoing content:

=for html
<blockquote>

I<"... I have been experimenting with all the different Apache compression modules
I have been able to find, but have not been able to get the desired result.
I have tried Apache::GzipChain, Apache::Compress, mod_gzip and mod_deflate, with
different results.  One I cannot get to work at all. Most work, but seem to collect
all the output before compressing it and sending it to the browser...>

I<... Wouldn't it be nice to have some option to specify that the handler should flush
and send the currently compressed output every time it had received a certain amount
of input or every time it had generated a certain amount of output?..>

I<... So I am basically looking for anyone who has had any success in achieving this
kind of "streaming" compression, who could direct me at an appropriate Apache module.">

=for html
</blockquote>

Unfortunately for him, C<Apache::Dynagzip> has not yet been publicly available at that time...

Since relesed this handler is especially useful when one needs to compress the outgoing
web content that is dynamically generated on the fly using templates,
DB data, XML, etc., and when at the time of the request it is impossible
to determine the length of the response.

Content provider can benefit additionally from the fact that handler begins the transmission
of compressed data concurent to further document creation.
On the other hand, the internal buffer inside the
handler prevents Apache from the creation of too short chunks over C<HTTP/1.1>.

In order to simplify the use of this handler on public/open-source sites,
the capability of content compression over HTTP/1.0 was added to this handler since the version 0.06.
This helps to avoid dynamic invocation of other Apache handlers
for the content generation phase.

=head2 Acknowledgments

Thanks to Tom Evans, Valerio Paolini, and Serge Bizyayev for their valuable idea contributions and multiple testing.
Thanks to Igor Sysoev and Henrik Nordstrom those helped me to understand better the HTTP/1.0 compression features.
Thanks to Vlad Jebelev for the patch that helps to survive possible dynamical Apache downgrade
from HTTP/1.1 to HTTP/1.0 (especially serving MSIE requests over SSL).