view release on metacpan or  search on metacpan

lib/Mojolicious/Guides/Cookbook.pod  view on Meta::CPAN

  my $url = Mojo::URL->new('http://fastapi.metacpan.org/v1/release/_search');
  $url->query({q => 'mojolicious', sort => 'date:desc'});
  for my $hit (@{$ua->get($url)->result->json->{hits}{hits}}) {
    say "$hit->{_source}{name} ($hit->{_source}{author})";
  }

=head2 Basic authentication

You can just add username and password to the URL, an C<Authorization> header will be automatically generated.

  use Mojo::UserAgent;

  my $ua = Mojo::UserAgent->new;
  say $ua->get('https://sri:secret@example.com/hideout')->result->body;

If you're using L<Mojo::URL> to build the URL, be aware that the userinfo part will not be included if the object is
stringified. You'll have to pass the object itself to L<Mojo::UserAgent> or use L<Mojo::URL/"to_unsafe_string">.

  use Mojo::UserAgent;
  use Mojo::URL;

  my $ua  = Mojo::UserAgent->new;
  my $url = Mojo::URL->new('https://example.com/hideout')->userinfo('sri:secret');
  say $ua->get($url)->result->body;

=head2 Decorating follow-up requests

L<Mojo::UserAgent> can automatically follow redirects, the event L<Mojo::UserAgent/"start"> allows you direct access to
each transaction right after they have been initialized and before a connection gets associated with them.

  use Mojo::UserAgent;

  # User agent following up to 10 redirects
  my $ua = Mojo::UserAgent->new(max_redirects => 10);

  # Add a witty header to every request
  $ua->on(start => sub ($ua, $tx) {
    $tx->req->headers->header('X-Bender' => 'Bite my shiny metal ass!');
    say 'Request: ', $tx->req->url->clone->to_abs;
  });

  # Request that will most likely get redirected
  say 'Title: ', $ua->get('google.com')->result->dom->at('head > title')->text;

This even works for proxy C<CONNECT> requests.

=head2 Content generators

Content generators can be registered with L<Mojo::UserAgent::Transactor/"add_generator"> to generate the same type of
content repeatedly for multiple requests.

  use Mojo::UserAgent;
  use Mojo::Asset::File;

  # Add "stream" generator
  my $ua = Mojo::UserAgent->new;
  $ua->transactor->add_generator(stream => sub ($transactor, $tx, $path) {
    $tx->req->content->asset(Mojo::Asset::File->new(path => $path));
  });

  # Send multiple files streaming via PUT and POST
  $ua->put('http://example.com/upload'  => stream => '/home/sri/mojo.png');
  $ua->post('http://example.com/upload' => stream => '/home/sri/minion.png');

The C<json>, C<form> and C<multipart> content generators are always available.

  use Mojo::UserAgent;

  # Send "application/json" content via PATCH
  my $ua = Mojo::UserAgent->new;
  my $tx = $ua->patch('http://api.example.com' => json => {foo => 'bar'});

  # Send query parameters via GET
  my $tx2 = $ua->get('search.example.com' => form => {q => 'test'});

  # Send "application/x-www-form-urlencoded" content via POST
  my $tx3 = $ua->post('http://search.example.com' => form => {q => 'test'});

  # Send "multipart/form-data" content via PUT
  my $tx4 = $ua->put('upload.example.com' => form => {test => {content => 'Hello World!'}});

  # Send custom multipart content via PUT
  my $tx5 = $ua->put('api.example.com' => multipart => ['Hello', 'World!']);

For more information about available content generators see also L<Mojo::UserAgent::Transactor/"tx">.

=head2 Large file downloads

When downloading large files with L<Mojo::UserAgent> you don't have to worry about memory usage at all, because it will
automatically stream everything above 250KiB into a temporary file, which can then be moved into a permanent file with
L<Mojo::Message/"save_to">.

  use Mojo::UserAgent;

  # Fetch the latest Mojolicious tarball
  my $ua = Mojo::UserAgent->new(max_redirects => 5);
  my $tx = $ua->get('https://www.github.com/mojolicious/mojo/tarball/main');
  $tx->result->save_to('mojo.tar.gz');

With L<Mojo::File/"download"> you can also stream file downloads directly into the target file. This allows for
interrupted downloads to be resumed at a later time.

  use Mojo::File qw(path);

  # Download the latest Mojolicious tarball directly into a file
  path('/home/sri/mojo.tar.gz')->download('https://www.github.com/mojolicious/mojo/tarball/main');

To protect you from excessively large files there is also a limit of 2GiB by default, which you can tweak with the
attribute L<Mojo::UserAgent/"max_response_size">.

  # Increase limit to 10GiB
  $ua->max_response_size(10737418240);

=head2 Large file upload

Uploading a large file is even easier.

  use Mojo::UserAgent;

  # Upload file via POST and "multipart/form-data"
  my $ua = Mojo::UserAgent->new;
  $ua->post('example.com/upload' => form => {image => {file => '/home/sri/hello.png'}});

And once again you don't have to worry about memory usage, all data will be streamed directly from the file.

=head2 Streaming response

Receiving a streaming response can be really tricky in most HTTP clients, but L<Mojo::UserAgent> makes it actually
easy.

  use Mojo::UserAgent;

  # Accept responses of indefinite size
  my $ua = Mojo::UserAgent->new(max_response_size => 0);

  # Build a normal transaction
  my $tx = $ua->build_tx(GET => 'http://example.com');

  # Replace "read" events to disable default content parser
  $tx->res->content->unsubscribe('read')->on(read => sub ($content, $bytes) {
    say "Streaming: $bytes";
  });

  # Process transaction
  $tx = $ua->start($tx);

The event L<Mojo::Content/"read"> will be emitted for every chunk of data that is received, even chunked transfer
encoding and gzip content encoding will be handled transparently if necessary.

=head2 Streaming request

Sending a streaming request is almost just as easy.

  use Mojo::UserAgent;

  # Build a normal transaction
  my $ua = Mojo::UserAgent->new;
  my $tx = $ua->build_tx(POST => 'http://example.com');

  # Prepare body
  my $body = 'Hello World!';
  $tx->req->headers->content_length(length $body);

  # Start writing directly with a drain callback
  my $drain = sub ($content) {
    my $chunk = substr $body, 0, 1, '';
    $content->write($chunk, length $body ? __SUB__ : undef);
  };
  $tx->req->content->$drain;

  # Process transaction
  $tx = $ua->start($tx);

The drain callback passed to L<Mojo::Content/"write"> will be executed whenever the entire previous chunk of data has
actually been written.

=head2 Non-blocking

L<Mojo::UserAgent> has been designed from the ground up to be non-blocking, the whole blocking API is just a simple
convenience wrapper. Especially for high latency tasks like web crawling this can be extremely useful, because you can
keep many concurrent connections active at the same time.

  use Mojo::UserAgent;
  use Mojo::IOLoop;

  # Concurrent non-blocking requests
  my $ua = Mojo::UserAgent->new;
  $ua->get('https://metacpan.org/search?q=mojo' => sub ($ua, $mojo) {
    say $mojo->result->dom->at('title')->text;
  });
  $ua->get('https://metacpan.org/search?q=minion' => sub ($ua, $minion) {
    say $minion->result->dom->at('title')->text;
  });

  # Start event loop if necessary
  Mojo::IOLoop->start unless Mojo::IOLoop->is_running;

But don't try to open too many connections to one server at the same time, it might get overwhelmed. Better use a queue
to process requests in smaller batches.

  use Mojo::Promise;
  use Mojo::UserAgent;

  my @urls = (
    'https://docs.mojolicious.org/Mojo/DOM',  'https://docs.mojolicious.org/Mojo',
    'https://docs.mojolicious.org/Mojo/File', 'https://docs.mojolicious.org/Mojo/URL'
  );

  # User agent with a custom name, following up to 5 redirects
  my $ua = Mojo::UserAgent->new(max_redirects => 5);
  $ua->transactor->name('MyParallelCrawler 1.0');