view release on metacpan or  search on metacpan

lib/HTTP/MultiGet.pm  view on Meta::CPAN

    my $request=$requests[$id++];
    print "Results for: ".$request->uri."\n";
    if($response->is_success) {
      print $response->decoded_content;
    } else {
      print $response->status_line,"\n";
    }
  }


=head2 Handling Multiple Large Downloads

  use Modern::Perl;
  use HTTP::MultiGet;
  use HTTP::Request;

  my $req=HTTP::Request->new(GET=>'http://some.realy.big/file/to/download.gz');
  my $req_b=HTTP::Request->new(GET=>'http://some.realy.big/file/to/download2.gz');

  # create a callback 
  my $code=sub {
    my ($getter,$request,$headers,$chunk)=@_;
      # 0: Current HTTP::MultiGet instance
      # 1: HTTP::Request object
      # 2: HTTP::Headers object
      # 3: Chunk of data being downloaded
    if($headers->header('Status')==200) {
      # do something
    } else {
      # do something with $body
    }
  };
  my $getter=new HTTP::MultiGet;
  my ($result,$result_b)=$getter->run_requests([$req,on_body=>$code],[$req_b,on_body=>$code]);


The arguments: on_body=>$code are called called on each chunk downloaded.  $result is created when the download is completed, but $result->decoded_content is going to be empty

=head1 DESCRIPTION

Created a wrapper for: L<AnyEvent::HTTP>, but provides a more LWP like feel.

=cut

use Moo;
use Log::Log4perl;
use Data::Queue;
use Scalar::Util qw(looks_like_number);
Log::Log4perl->wrapper_register(__PACKAGE__);
use AnyEvent;
use Data::Dumper;
use HTTP::Response;
use HTTP::Headers;
use HTTP::Request;
use Class::Method::Modifiers;
use AnyEvent::HTTP::Request;
use MooX::Types::MooseLike::Base qw(:all);
use AnyEvent::HTTP::Response;
use AnyEvent;
use Carp qw(croak);
use Ref::Util qw(is_plain_arrayref is_plain_hashref);
require AnyEvent::HTTP;
use namespace::clean;
BEGIN {
with 'Log::LogMethods';
with 'Data::Result::Moo';
}
our $VERSION='1.023';

sub BUILD {
  my ($self)=@_;

  $self->{stack}=new Data::Queue;
  $self->{que_count}=0;
  $self->{que_id}=0;
}

=head1 Moo Stuff

This is a Moo class the object constructor takes the following arguments, along with the following roles

Role List:

  Log::LogMethods
  Data:::Result::Moo

Arguemnts and object accessors:

  logger:          DOES(Log::Log4perl::Logger)
  request_opts:    See AnyEvent::HTTP params for details
  timeout:         Global timeout for everything ( default 300 )
  max_que_count:   How many requests to run at once ( default 20 )
  max_retry:       How many times to retry if we get a connection/negotiation error 

For internal use only: 

  in_control_loop: true when in the control loop
  stack:           Data::Queue object 
  que_count:       Total Number of elements active in the que
  retry:           Anonymous hash used to map ids to retry counts

=head2 UNIT TESTING

For Unit testing

  on_create_request_cb: Anonymous code ref to be called 
    when a new request object has been created
        sub { my ($id,$request)=@_ }

Arguments  for the call back

  id:  number for the object
  req: a new instance of $self->SENDER_CLASS

Interal blocking control variables

  loop_control: AnyEvent->condvar object
  false_id: internal false id tracker
  fake_jobs: Internal object for handling fake results

=cut