Acme-CPANModulesBundle-Import-PerlDancerAdvent-2018
view release on metacpan or search on metacpan
"generated_by" : "Dist::Zilla version 6.010, CPAN::Meta::Converter version 2.150010",
"license" : [
"perl_5"
],
"meta-spec" : {
"url" : "http://search.cpan.org/perldoc?CPAN::Meta::Spec",
"version" : 2
},
"name" : "Acme-CPANModulesBundle-Import-PerlDancerAdvent-2018",
"prereqs" : {
"configure" : {
"requires" : {
"ExtUtils::MakeMaker" : "0"
}
},
"develop" : {
"requires" : {
"Pod::Coverage::TrustPod" : "0",
"Test::Perl::Critic" : "0",
"Test::Pod" : "1.41",
"Test::Pod::Coverage" : "1.08"
---
abstract: 'Modules mentioned in PerlDancer Advent Calendar 2018'
author:
- 'perlancar <perlancar@cpan.org>'
build_requires:
File::Spec: '0'
IO::Handle: '0'
IPC::Open3: '0'
Test::More: '0'
configure_requires:
ExtUtils::MakeMaker: '0'
dynamic_config: 0
generated_by: 'Dist::Zilla version 6.010, CPAN::Meta::Converter version 2.150010'
license: perl
meta-spec:
url: http://module-build.sourceforge.net/META-spec-v1.4.html
version: '1.4'
name: Acme-CPANModulesBundle-Import-PerlDancerAdvent-2018
resources:
bugtracker: https://rt.cpan.org/Public/Dist/Display.html?Name=Acme-CPANModulesBundle-Import-PerlDancerAdvent-2018
devdata/http_advent.perldancer.org_2018_16 view on Meta::CPAN
$ENV{ MOJO_PUBSUB_EXPERIMENTAL } = 1;
Minion->new( mysql =>
MyJob::DBConnectionManager->new->get_connection_uri({
db_type => 'feeds',
io_type => 'rw',
}));
},
);</pre>
<p>We wrapped a simple Moose class around Minion to make it easy to add to any class or Dancer application with the extra functionality we wanted.</p>
<p>We ran into an issue at one point where jobs weren't running since we added them to a queue that no worker was configured to handle. To prevent this from happening to us again,
we added code to prevent us from adding code to a queue that didn't exist:</p>
<pre class="prettyprint">my @QUEUE_TYPES = qw( default InstantXML PayrollXML ChangeRequest );
sub has_invalid_queues( $self, @queues ) {
return 1 if $self->get_invalid_queues( @queues );
return 0;
}
sub get_invalid_queues( $self, @queues ) {
my %queue_map;
devdata/http_advent.perldancer.org_2018_16 view on Meta::CPAN
name => "InstantXML",
guid => $guid,
title => "Instant XML Generator",
queue => 'InstantXML',
job_args => [ $self->request_path, $guid, $group, $force ],
});
}</pre>
<h2><a name="creating_and_configuring_the_job_queue_worker"></a>Creating and Configuring the Job Queue Worker</h2>
<p>We wanted to easily configure our Minions for all hosts and environments in one spot. Since we use a lot of YAML in Dancer, specifying the Minion configuration in YAML made a lot of sense
to us:</p>
<pre class="prettyprint"># What port does the dashboard listen on?
dashboard_port: 4000
# Add the rest later.
dashboards:
UNKNOWN: http://localhost:3000/
DEV: http://my.development.host.tld:8001/
# Hosts that have no entry assume the default configuration
devdata/http_advent.perldancer.org_2018_16 view on Meta::CPAN
$logger->error( $error );
});
});</pre>
<p>To help us for future capacity planning, we want our workers to tell us if they are running at peak capacity, so log when this event occurs:</p>
<pre class="prettyprint">$worker->on( busy => sub( $worker ) {
my $max = $worker->status->{ jobs };
$logger->log( "$0: Running at capacity (performing $max jobs)." );
});</pre>
<p>Now, we apply the configuration (read below) to the worker. When the worker starts, it tells us information about how it was configured (this was really useful during development):</p>
<pre class="prettyprint">my $max_jobs = $hostconfig->{ max_children };
my @queues = @{ $hostconfig->{ queues }};
if( $minion->has_invalid_queues( @queues ) ){
print "Invalid job queues specified: " . join( ',',
$minion->get_invalid_queues( @queues ) );
say ". Aborting!";
exit 1;
}
say "Starting Job Queue Worker on " . get_hostname();
say "- Configured to run a max of $max_jobs jobs";
say "- Listening for jobs on queues: ", join(', ', @queues );
$worker->status->{ jobs } = $max_jobs;
$worker->status->{ queues } = \@queues;
$worker->run;</pre>
<p>Remember the YAML file we used to configure things up above? This last bit pulls the information for the host this worker is running on (<code>get_hostname()</code> is a home-grown
hostname function):</p>
<pre class="prettyprint">sub get_hostconfig {
my $minion_config =
MyJob::Config->new({ filename => "environments/minions.yml" })->config;
my $hostname = get_hostname();
if( exists $minion_config->{ $hostname }) {
return $minion_config->{ $hostname };
} else {
return $minion_config->{ default };
devdata/http_advent.perldancer.org_2018_19 view on Meta::CPAN
<p>There are a couple of important nuances you should be aware of:</p>
<ul>
<li><a name="item_Environment_configuration_replaces_logging_configuration"></a><b>Environment configuration replaces logging configuration</b>
<p>If you put your logging configuration in <i>config.yml</i> rather than one of your environment-specific
configuration files (such as <i>environments/development.yml</i>), you stand a good chance of not using
the logging configuration you think you are using. The default configuration file for the development
environment, for example, logs only to the console. If you put your Log4perl configuration in <i>config.yml</i>
and don't change your development configuration file, your Log4perl configuration will be passed over
for the default console logger.</p>
<p>From my own experience, <b>always</b> configure your logger in your environment-specific configuration, unless
you use the same configuration across all environments (I don't endorse this practice).</p>
</li>
<li><a name="item_Core_level_messages_are_passed_as_log_level_trace__but_will_not_be_passed_unless_Dancer2_s_log_level_is_core_"></a><b>Core level messages are passed as log level trace, but will not be passed unless Dancer2's log level is core.</b>
<p>Since <code>core</code> doesn't have a good corresponding level in Log4perl, <code>core</code> level messages are sent
over to Log4perl at the <code>trace</code> log level. This <b>only</b> happens when you set Dancer2's log level in your
<i>config.yml</i> file to <code>core</code> however. So your preferred log level setting is respected, even if <code>core</code> level
messages have to be reported at a different level.</p>
</li>
<li><a name="item__code_log__code__should_be_set_a_lower_priority_than_the_lowest_priority_as_set_in_your_Log4perl_configuration_"></a><b><code>log</code> should be set a lower priority than the lowest priority as set in your Log4perl configuration.<...
<p>If it isn't, the log messages will not be passed to Log4perl.</p>
( run in 0.513 second using v1.01-cache-2.11-cpan-0a6323c29d9 )