Acme-CPANModulesBundle-Import-MojoliciousAdvent-2017
view release on metacpan or search on metacpan
devdata/https_mojolicious.io_blog_2017_12_02_day-2-the-stash view on Meta::CPAN
</div>
<div class="post-thumb">
<!-- theme suggests 1300x500 -->
<img alt="leather bag" src="/blog/2017/12/02/day-2-the-stash/bag-1854148_1920.jpg">
</div>
<div class="post-content">
<section id="section-1">
<p>In Mojolicious, when processing a request and preparing a response one of the most important concepts is "the stash".
Since it is a non-blocking framework, your code can't use global variables to store any state during processing.
If you did and some other code were to run, it could very easily get cross-talk between requests.</p>
<p>The stash is the place you can store information while you process it.
It is just a simple hash reference that is attached to the controller object that is processing the request.
It lives and dies with that one transaction.</p>
<p>While you can and should use it as a scratchpad, it really is much more.
The stash controls almost every aspect of the response that you generate.
Let's look a little closer to see how it works</p>
</section>
<section id="section-2">
<h2>Using the Stash for Rendering Text</h2>
devdata/https_mojolicious.io_blog_2017_12_04_day-4-dont-fear-the-full-app view on Meta::CPAN
Nothing could be further from the truth.
<a href="http://mojolicious.org/perldoc/Mojolicious/Lite">Mojolicious::Lite</a> is a very tiny wrapper around the so-called "Full" app architecture, giving it the approachable keyword syntax.</p>
<p>Because it is much nicer to have concise single-file examples for documentation most of Mojolicious' documentation uses Lite syntax most of the time.
It is understandable that people worry about migrating (or as we call it "growing") even once their apps would benefit from Object-Oriented structure; after all the docs seem geared towards Lite apps.
However, let those fears go, the transition is easy.
And once you understand it, the documentatation examples are trivial to translate.</p>
<p>Plus, Mojolicious comes with two forms of help when transitioning.
The first is the <a href="http://mojolicious.org/perldoc/Mojolicious/Guides/Growing">Growing Guide</a> which covers everything this post will but from the perspective of porting an existing application (which I won't duplicate here).
The second is the <a href="http://mojolicious.org/perldoc/Mojolicious/Command/inflate">inflate command</a> which can even start you on the process by moving your templates from the data section and into their own files.</p>
<p>That said, in order to futher demystify things, I'm going to cover some of the differences and pull back the curtain on the Lite syntax itself.</p>
</section>
<section id="section-2">
<h2>Let Me Convince You</h2>
<p>After repeated attempts to convince people that there is very little difference between the two, I've found that there is one really great way to turn the conversation.
I show them the code.
No really <a href="https://github.com/kraih/mojo/blob/master/lib/Mojolicious/Lite.pm">take a look</a>.
devdata/https_mojolicious.io_blog_2017_12_05_day-5-your-apps-built-in-commands view on Meta::CPAN
<p>This output includes all the same stuff as before but this time it also adds a few extra items.
Certain routes are more complex, and while all these were simple and so no flags are shown, if one were an <code>under</code> route or a <code>websocket</code> it would be noted where the <code>....</code> are.
Finally it includes a pattern that is what is actually matched by the router.
This can be helpful sometimes when debugging why certain requests match (or more likely don't match) a certain route.
Note that the router checks each route in order top to bottom, the first to match is what is used.</p>
<h3>The get Command</h3>
<p>Now we're getting to the fun stuff!</p>
<p>Mojolicious comes with a <a href="http://mojolicious.org/perldoc/Mojo/UserAgent">user agent</a> and lots of post-processing power, including <a href="http://mojolicious.org/perldoc/Mojo/DOM">HTML/XML</a> and <a href="http://mojolicious.org/perldoc...
This command exposes those features together on the command line, like a smart version of cURL or wget.</p>
<p>Output is written to STDOUT so that you can redirect the result to a file if you'd like.
Because of that, headers are omitted from the output unless you pass <code>-v</code>.</p>
<p>Let's see what it can do!
You can find the latest version of <code>IO::Socket::SSL</code> using the <a href="https://github.com/metacpan/metacpan-api/blob/master/docs/API-docs.md">Meta::CPAN JSON API</a>.
The response is parsed as JSON and only the <code>version</code> element is output.</p>
<pre><code>mojo get https://fastapi.metacpan.org/v1/module/IO::Socket::SSL /version
devdata/https_mojolicious.io_blog_2017_12_06_day-6-adding-your-own-commands view on Meta::CPAN
<p>Programmers hate duplication because of skew.
If code gets improved in one place, it is unlikely to be improved in all places, unless there is only the one.
So that script you wrote a while back, the one with the database connection you hand-rolled, is that still correct?</p>
<p>In the <a href="/blog/2017/12/05/day-5-your-apps-built-in-commands">previous article in this series</a> I talked about the built-in commands available to your application.
The final command was <a href="http://mojolicious.org/perldoc/Mojolicious/Command/eval"><code>eval</code></a>.
I mentioned that when combined with predefined behaviors, the command could be great for administrative tasks.
That's true, but you need to know what to eval in order to do so.</p>
<p>To formalize that process, we can go one step further: defining our own commands.
By doing this your application's administative behaviors can take arguemnts and provide optional switches as well as give usage messages.
In this way these administative commands decouple themselves from knowledge of the application's internals and become useful to a broader set of users.</p>
</section>
<section id="section-2">
<h2>What is a Command?</h2>
<p>Structurally, a command is just a class that inherits from <a href="http://mojolicious.org/perldoc/Mojolicious/Command">Mojolicious::Command</a> and implements a <code>run</code> method.
The method is passed an instance of the command class and the arguments given on the command line.
The command has the application as an attribute.
devdata/https_mojolicious.io_blog_2017_12_12_day-12-more-than-a-base-class view on Meta::CPAN
<p>becomes</p>
<pre><code>use Mojo::Base -strict, -signatures;
$ua->get('/url' => sub ($ua, $tx) { ... });
</code></pre>
<h2>Establishing a Class</h2>
<p>Now for the functionality that the name implies, setting up a class.
The Mojo stack modules make LOTS of objects in the course of performing their tasks, from making or handling HTTP requests to processing HTML documents, object creation time is key.
Therefore the primary feature of Mojo::Base classes is speed!</p>
<p>The object system is spartan even when compared to Moo.
You get a hash-based object with declarative read/write lazy accessors and a constructor.
When considering Moose vs Moo, you trade lesser-used features for a noticable performance gain.
Likewise when you consider Mojo::Base vs Moo, you strip down futher, this time to the bare essentials, but again get a performance gain.</p>
<p>Now if you need more functionality, you are more than welcome to use Moo or other object systems in your applications (though the Mojo internals will of course continue to use Mojo::Base).
That said, much of the real world usage of Moo is very similar to Mojo::Base: lazy accessor generation; this might be all you need.</p>
devdata/https_mojolicious.io_blog_2017_12_13_day-13-more-about-roles view on Meta::CPAN
$t->get_ok('/')
->text_deeply(
'nav a',
[qw( Home Blog Projects About Contact )],
'nav link text matches site section titles',
)->d(sub{ diag $t->tx->req->body });
</code></pre>
<p>This example loads a hypothetical psgi application from the filesystem, tests it for text elements and if that test fails dumps the response body to the console.</p>
<p>There are three modules (only two on CPAN so far) that let you run javascript on pages via external processes.
<a href="https://metacpan.org/pod/Test::Mojo::Role::Phantom">Test::Mojo::Role::Phantom</a> and <a href="https://metacpan.org/pod/Test::Mojo::Role::Selenium">Test::Mojo::Role::Selenium</a> use <a href="http://phantomjs.org/">PhantomJS</a> and <a href=...
PhantomJS is abandoned however so <a href="https://github.com/jberger/Mojo-Chrome">Test::Mojo::Role::Chrome</a> is in development on my Github and will replace the PhantomJS role.</p>
<p>Note that several of these modules depend on a module named <a href="https://metacpan.org/pod/Test::Mojo::WithRoles">Test::Mojo::WithRoles</a> which is essentially obsolete at this point.
It can be safely ignored and authors can remove it from their module's dependencies when convenient.</p>
<p>Apart from testing, other roles include</p>
<ul>
<li><a href="https://metacpan.org/pod/Mojo::Log::Role::Clearable">Mojo::Role::Log::Clearable</a> allowing the Mojo::Log to cleanly change paths</li>
<li><a href="https://metacpan.org/pod/Mojo::DOM::Role::PrettyPrinter">Mojo::DOM::Role::PrettyPrinter</a> pretty prints Mojo::DOM documents</li>
<li><a href="https://metacpan.org/pod/Mojo::Collection::Role::UtilsBy">Mojo::Collection::Role::UtilsBy</a> adds <a href="https://metacpan.org/pod/List::UtilsBy">List::UtilsBy</a> methods to <a href="http://mojolicious.org/perldoc/Mojo/Collection">Moj...
<li><a href="https://metacpan.org/pod/distribution/Mojo-UserAgent-CookieJar-Role-Persistent/lib/Mojo/UserAgent/CookieJar/Role/Persistent.pod">Mojo::UserAgent::CookieJar::Role::Persistent</a> can store cookies from Mojo::UserAgent requests persistentl...
</ul>
<p>... as well as others.
And the list is sure to grow now that Mojo::Base can natively compose these roles.</p>
</section>
<small><p><a href="https://commons.wikimedia.org/w/index.php?curid=15179120">Image</a> © Jorge Royan / <a class="external free" href="http://www.royan.com.ar" rel="nofollow">http://www.royan.com.ar</a>, <a href="https://creativ...
</small>
<p class="tags">
devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call view on Meta::CPAN
<section id="section-1">
<p>A new feature of <a href="http://mojolicious.org/">Mojolicious</a>, as of <a href="https://metacpan.org/release/SRI/Mojolicious-7.49">7.49</a>, is the implementation of the <a href="https://promisesaplus.com/implementations#in-ot...
</section>
<section id="section-2">
<h2>Background</h2>
<p>"Normal" Perl code runs synchronously: it does each step it is told to, one at a time, and only that. This is also known as "blocking", since the program cannot do anything else.</p>
<p>The essence of a non-blocking code framework is that if you are waiting for something, you can register with the framework what to do when that thing happens. It can then do other processing tasks in the meantime. This means you don't have lot...
<p>Originally this was done just using callbacks, but this lead to what is known as "callback hell": each callback contains the next callback, at an increasing level of indentation. Even harder to keep track of is if the functions are kept ...
<p>Promises are used to easily add processing steps to a transaction: one can keep adding code for what to do "then" - after a previous stage has finished. Best of all, each "callback" is small and separate, with each one placed i...
<p>First let's get web pages, one after the other, synchronously. Obviously, that means the code will block anything else while it's running.</p>
<pre><code># refers to a previously-set-up @urls
sub fetchpages {
while (my $url = shift @urls) {
# Fetch, show title
say $ua->get($url)->result->dom->at('title')->text;
}
}
devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call view on Meta::CPAN
$ua->get($url, sub {
my ($tx) = @_;
say "$url: ", $tx->result->dom->at('title')->text;
fetchpages();
});
}
</code></pre>
<h2>Promises</h2>
<p>With promises, we're going to split the processing, still in a single "stream" of activity, into two steps, to show the use of <code>then</code>. Notice the first <code>then</code> doesn't return a Promise, just a normal object. ...
<pre><code>sub fetchpages {
# Stop if there are no more URLs
return unless my $url = shift @urls;
# Fetch the next title
$ua->get_p($url)->then(sub {
my ($tx) = @_;
$tx->result;
})->then(sub {
my ($result) = @_;
say "$url: ", $result->dom->at('title')->text;
fetchpages(); # returns a promise, but of a whole new page to process
});
}
</code></pre>
<p>Here you'll see we're using <a href="http://mojolicious.org/perldoc/Mojo/UserAgent#get_p"><code>get_p</code></a>. This is just like <a href="http://mojolicious.org/perldoc/Mojo/UserAgent#get"><code>get</code></a>, but instead of taking a c...
<h2>Promises with two streams</h2>
<p>Given that a Promise is a single chain of processing steps, how can we have a number of them running concurrently, without making all the requests at once? We'll use two ideas: chaining (shown above - the key is each "then" returns a...
<pre><code>sub fetchpages {
# Stop if there are no more URLs
return unless my $url = shift @urls;
# Fetch the next title
$ua->get_p($url)->then(sub {
my ($tx) = @_;
$tx->result;
})->then(sub {
my ($result) = @_;
say "$url: ", $result->dom->at('title')->text;
fetchpages(); # returns a promise, but of a whole new page to process
});
}
# Process two requests at a time
my @promises = map fetchpages(), 1 .. 2;
Mojo::Promise->all(@promises)->wait if @promises;
</code></pre>
<p>Another option for dealing with a number of concurrent activities, if you just want the first one that completes, is <a href="http://mojolicious.org/perldoc/Mojo/Promise#race"><code>race</code></a>.</p>
devdata/https_mojolicious.io_blog_2017_12_14_day-14-you-promised-to-call view on Meta::CPAN
my $url = $urlbase . $s;
print "getting $url\n";
$ua->get_p($url)->then(sub {
my ($tx) = @_;
handle_result($outpath, $tx, $s);
makepromise($urlbase, $ua, $suffixes, $outpath);
});
}
</code></pre>
<p>Once each stream runs out of suffixes to process, it will finish. If we wanted to add the ability to add to the queue that could keep as many streams as we started, we would restructure so that each stream is subscribed to a queue, and if the queu...
<h2>See also</h2>
<ul>
<li>The Mojolicious Cookbook shows how to implement non-blocking requests <a href="http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Concurrent-blocking-requests">with promises</a>.</li>
<li>The new <a href="http://mojolicious.org/perldoc/Mojo/Promise">Mojo::Promise</a> class documentation.</li>
<li>This script is now available as a <code>Mojolicious::Command</code>: <a href="https://metacpan.org/pod/Mojolicious::Command::bulkget">Mojolicious::Command::bulkget</a>!</li>
</ul>
</section>
devdata/https_mojolicious.io_blog_2017_12_16_day-16-the-secret-life-of-sessions view on Meta::CPAN
<span class="hljs-type">$message</span> .= '<span class="hljs-string">s</span>' <span class="hljs-keyword">unless</span> <span class="hljs-type">$count</span> == <span class="hljs-number">1</span>;
<span class="hljs-type">$c</span>-><span class="hljs-type">render</span>(text => <span class="hljs-type">$message</span>);
};
app->start;
</code></pre>
<p>So why isn't this recommended?
Because it would mean that each time you started the server you would get a new secret.
As with the case of changing your secret intentionally above, all existing sessions would be invalidated any time you wanted to reboot a server or restart the server process.
Additionally you could only use one server, any load balancing scenario would result in different secrets on different hosts, your users would randomly invalidate their sessions depending on which server they hit!</p>
<h2>Forward Secrecy</h2>
<p>Just as you have to change application passwords from time to time, so too you need to change your secret.
In a brute force scenario, a nefarious user could take one of their cookies and try to reverse engineer the secret that was used to generate it.</p>
<p>But wait!
You just said that changing the secret to prevent brute forcing will invalidate all of our sessions!</p>
devdata/https_mojolicious.io_blog_2017_12_16_day-16-the-secret-life-of-sessions view on Meta::CPAN
Don't worry, as long as it said that it started it should stay running.
To stop the application, run the same commmand but also pass a <code>-s</code> switch</p>
<pre><code>$ hypnotoad -s myapp.pl
</code></pre>
<p>The really interesting thing happens when restarting a running application.
Say you've rolled your secrets and want to restart the application to use it.
Simply run the command as before, just like when you started it up.</p>
<p>Any requests currently being served by the old server processes will continue to be served by them (up to some timeout).
Any new requests will be served by new processes using the new configuration.
You don't cut off any users in the process.</p>
<p>Actually <code>hypnotoad</code> can be used to zero downtime restart for more reasons than just configuration changes.
It can handle changes to your application, updating dependencies, or even upgrading your version of Perl itself!</p>
<p>For now though, we're just satisfied that clients are none the wiser that you rolled the application secret out from under them without losing any connections nor invalidating any sessions.</p>
</section>
<small><p><a href="https://commons.wikimedia.org/w/index.php?curid=54930070">Image</a> by <a class="external text" href="https://www.flickr.com/people/30478819@N08" rel="nofollow">Marco Verch</a> - <a class="external text" href="https:/...
</small>
devdata/https_mojolicious.io_blog_2017_12_17_day-17-the-wishlist-app view on Meta::CPAN
</code></pre>
<p><small>templates/login.html.ep</small></p>
<p>Immediately you can see that there are a few statements at the top.
These set the layout to our default one above and set the title key for the page.
It is important to realize that this page is rendered <strong>first</strong> before any layout is rendered.</p>
<p>After the template renders, Mojolicious will notice that a layout key was set and as a result it will render the layout with the result of the primary template rendering available as the default content buffer.
As we saw before, that content will be placed inside the <code>#main</code> section of the page.
However, in the process of rendering the primary template, the title was also set.
Since the layout is rendered afterwards, this value is now available to set the <code><title></code> tag at the top of the page.</p>
<p>While this may seem obvious, it is actually quite remarkable.
If the page had been rendered all in order, the value would not have been set in time to be used there at the top of the page.
Knowing the rendering order is therefore very important to understanding the rendering process.</p>
<h2>The Application</h2>
<p>I've gone about as far as is practical without showing you, dear reader, what the actual application script looks like.</p>
<pre><code class="hljs"><span class="hljs-keyword">use</span> <span class="hljs-function">Mojolicious::Lite</span>;
<span class="hljs-keyword">use</span> <span class="hljs-function">DBM::Deep</span>;
<span class="hljs-keyword">use</span> LinkEmbedder;
devdata/https_mojolicious.io_blog_2017_12_18_day-18-the-wishlist-model view on Meta::CPAN
</section>
<section id="section-2">
<h2>Model View Controller</h2>
<p>Most modern web applications adhere to a pattern called <a href="https://en.wikipedia.org/wiki/Model%E2%80%93view%E2%80%93controller">Model View Controller</a> or MVC.
Much has been written about MVC, more than could be conveyed here.
Quickly though, the view is how data is displayed, the template.
The model is the database, both read and write access, and how to manipulated it.
This is usually thought of as the "business logic".
Finally the controller is supposed to be the minimal amount of logic that can be used to connect the two and process web requests.</p>
<p>In Perl almost all relational database access is via <a href="https://metacpan.org/pod/DBI">DBI</a> whether directly or indirectly.
However this isn't a model as such, a model really needs to have defined data layout (schema) and manipulation.</p>
<p>Many Perl users turn to <a href="https://metacpan.org/pod/DBIx::Class">DBIx::Class</a>, an <a href="https://en.wikipedia.org/wiki/Object-relational_mapping">Object-Relational Mapper</a> (ORM), and this is a great choice.
It hides most of the SQL and creates classes for the tables.
However some developers like staying a little closer to the SQL.</p>
<h2>Mojo-Flavored DBI</h2>
devdata/https_mojolicious.io_blog_2017_12_19_day-19-make-your-app-installable view on Meta::CPAN
Having an installable module also means that you can use a so-called "DarkPAN" and related tools to build yourself a local "CPAN".
If you have multiple Perl modules at your company (or as part of any project) using a DarkPAN can ease integration and deployment immensely!
N.B. there is even a DarkPAN tool written using Mojolicious called <a href="https://metacpan.org/pod/App::opan">opan</a>.</p>
<p>And hey if you needed even more more reason, it cleans up your project root directory somewhat too!</p>
</section>
<section id="section-2">
<h2>The Challenge</h2>
<p>The hardest part of this process is managing static files.
There are several Perl install tools that each handle static files slightly differently.
Some will happily bundle any files in the project directory, others will only bundle Perl files by default.</p>
<p>Then once installed that bundle location is usually different that what it was during development (even relative to the project).
It can be a challenge to make it so that you application can find those file both during development and after installation.</p>
<h2>TIMTOWTDI</h2>
<p>Before we get started, I should mention that there are many ways to accomplish this task.
If your application small and already located in a <code>script</code> directory and has no external static files, you're probably already done!
That isn't the case for most applications of any real size, however.</p>
<p>If you've read the <a href="http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook">Cookbook</a>, you've already seen that there is a section on <a href="http://mojolicious.org/perldoc/Mojolicious/Guides/Cookbook#Making-your-applicatio...
While that's true, it makes its recommendations without using external modules and for one specific installation tool.
It also assumes that your application <a href="http://mojolicious.org/perldoc/Mojolicious#home"><code>home</code></a> should be related to the installation directory, which isn't always the case.</p>
<p>Yours truly has even written a <a href="https://metacpan.org/pod/Mojolicious::Plugin::InstallablePaths">module</a> that was intended to ease this process somewhat.
While it does that, and I don't intend to deprecate it, I think there are even easier patterns now.</p>
<h2>The Share Directory</h2>
<p>Perl has a lesser known and somewhat informal functionality for bundling static files called a "share directory".
Create a directory in your project root called <code>share</code> which will serve for this purpose.
Then move the <code>templates</code> and <code>public</code> directories from your project root into that directory.
You should also move any other static files like say database migration files (e.g. <code>wishlist.sql</code> from <a href="https://mojolicious.io/blog/2017/12/18/day-18-the-wishlist-model/">yesterday</a>).</p>
<p>Although each install tool has different ways of specifying where the share directory is located during the development phase, none is espectially difficult to work with.
devdata/https_mojolicious.io_blog_2017_12_19_day-19-make-your-app-installable view on Meta::CPAN
}
</code></pre>
<p>Set these directories before the bundled versions so that if a file exists within them, they get used while defaulting to the bundled ones otherwise.</p>
<pre><code>if (-d $templates) {
unshift @{ $app->renderer->paths }, $templates;
}
</code></pre>
<p>Note that you could use that kind of process to allow other configured files to be relative to the home.
The application home is a great place to put data like a sqlite database; the sqlite database file seen earlier, which as show had to be an absolute path.
Indeed such a transform exists already exists in the Wishlist app but was omitted above for clarity.</p>
<h2>In Conclusion</h2>
<p>I have made some of these changes to the <a href="https://github.com/jberger/Wishlist/compare/blog_post/sqlite_model...blog_post/installable">Wishlist App</a>.
You'll see that it it really isn't much to do.</p>
<p>You also see that I only used as much of these instructions as I needed; you can do the same.
There is no magic in any of what you've seen (other than perhaps File::Share).
devdata/https_mojolicious.io_blog_2017_12_23_day-23-one-liners-for-fun-and-profit view on Meta::CPAN
<p>You can also use Perl's looping constructs.
Perhaps you have a file full of sites for which you want to get some data.</p>
<pre><code>$ cat sites
mojolicious.org
mojolicious.io
mojocasts.com
</code></pre>
<p>You can then take each line, request it, and post-processes.</p>
<pre><code>perl -Mojo -nlE 'say g($_)->dom->at("title")->text' sites
</code></pre>
<h3>Object Constructors</h3>
<p>The Mojo toolkit contains many helpful classes.
The ojo functions provide quick constructors for them, so you can access their functionality without all the typing!</p>
<ul>
( run in 0.523 second using v1.01-cache-2.11-cpan-8d75d55dd25 )