view release on metacpan or  search on metacpan

devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call  view on Meta::CPAN

            <div class="post-content">

              <section id="section-1">
                  <p>A new feature of <a href="http://mojolicious.org/">Mojolicious</a>, as of <a href="https://metacpan.org/release/SRI/Mojolicious-7.49">7.49</a>, is the implementation of the <a href="https://promisesaplus.com/implementations#in-ot...

              </section>
              <section id="section-2">
                  <h2>Background</h2>

<p>&quot;Normal&quot; Perl code runs synchronously: it does each step it is told to, one at a time, and only that. This is also known as &quot;blocking&quot;, since the program cannot do anything else.</p>

<p>The essence of a non-blocking code framework is that if you are waiting for something, you can register with the framework what to do when that thing happens. It can then do other processing tasks in the meantime. This means you don&#39;t have lot...

<p>Originally this was done just using callbacks, but this lead to what is known as &quot;callback hell&quot;: each callback contains the next callback, at an increasing level of indentation. Even harder to keep track of is if the functions are kept ...

<p>Promises are used to easily add processing steps to a transaction: one can keep adding code for what to do &quot;then&quot; - after a previous stage has finished. Best of all, each &quot;callback&quot; is small and separate, with each one placed i...

<p>First let&#39;s get web pages, one after the other, synchronously. Obviously, that means the code will block anything else while it&#39;s running.</p>

<pre><code># refers to a previously-set-up @urls
sub fetchpages {
  while (my $url = shift @urls) {
    # Fetch, show title
    say $ua-&gt;get($url)-&gt;result-&gt;dom-&gt;at(&#39;title&#39;)-&gt;text;
  }
}
</code></pre>

<h2>With a callback</h2>

<p>This you could realistically have running as part of a web service, since with any kind of callback it will run asynchronously, therefore non-blocking, as discussed above.</p>

<pre><code>sub fetchpages {
  # Stop if there are no more URLs
  return unless my $url = shift @urls;
  # Fetch the next title
  $ua-&gt;get($url, sub {
    my ($tx) = @_;
    say &quot;$url: &quot;, $tx-&gt;result-&gt;dom-&gt;at(&#39;title&#39;)-&gt;text;
    fetchpages();
  });

devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call  view on Meta::CPAN

  });
}
</code></pre>

<p>If either the initial <code>get_p</code>, or either of the <code>then</code>s get rejected, then execution will skip to the <code>catch</code>. Another way to get this behaviour is to give a second code-ref to <code>then</code>.</p>

<p><code>finally</code> is given a code-ref which will be called with either the successful (i.e. resolved) value, or the failure (i.e. the rejection) value.</p>

<h2>The task at hand</h2>

<p>We have to synchronise the work between the multiple &quot;streams&quot; of execution, so that nothing gets missed, or done twice. Luckily, in the asynchronous but single-threaded context we have here, we can just pass around a reference to a sing...

<pre><code>#!/usr/bin/env perl

# cut down from https://stackoverflow.com/questions/15152633/perl-mojo-and-json-for-simultaneous-requests/15166898#15166898
sub usage { die &quot;Usage: bulkget-delay urlbase outdir suffixesfile\n&quot;, @_ };
# each line of suffixesfile is a suffix
# it gets appended to urlbase, then requested non-blocking
# output in outdir with suffix as filename

use Mojo::Base -strict;

devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call  view on Meta::CPAN

  my $url = $urlbase . $s;
  print "getting $url\n";
  $ua->get_p($url)->then(sub {
    my ($tx) = @_;
    handle_result($outpath, $tx, $s);
    makepromise($urlbase, $ua, $suffixes, $outpath);
  });
}
</code></pre>

<p>Once each stream runs out of suffixes to process, it will finish. If we wanted to add the ability to add to the queue that could keep as many streams as we started, we would restructure so that each stream is subscribed to a queue, and if the queu...

<h2>See also</h2>

<ul>
<li>The Mojolicious Cookbook shows how to implement non-blocking requests <a href="http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Concurrent-blocking-requests">with promises</a>.</li>
<li>The new <a href="http://mojolicious.org/perldoc/Mojo/Promise">Mojo::Promise</a> class documentation.</li>
<li>This script is now available as a <code>Mojolicious::Command</code>: <a href="https://metacpan.org/pod/Mojolicious::Command::bulkget">Mojolicious::Command::bulkget</a>!</li>
</ul>

              </section>