Parallel-ForkManager-Scaled
view release on metacpan or search on metacpan
# In the child process now
# do some work ..
# Exit the child
$pm->finish;
}
DESCRIPTION
This module inherits from Parallel::ForkManager and adds the ability to
automatically manage the number of processes running based on how busy
the system is by watching the CPU idle time. Each time a child is about
to be start()ed a new value for max_procs may be calculated (if enough
time has passed since the last calculation). If a new value is
calculated, the number of processes to run will be adjusted by calling
set_max_procs with the new value.
Without specifying any attributes to the constructor, some defaults
will be set for you (see Attributes below).
Attributes
soft_min_procs
soft_max_procs
This is initially set to hard_min_procs and hard_max_procs
respectively and is adjusted over time. These are used when
calculating adjustments as the minimum and maximum number of
processes respectively.
Over time soft_min_procs and soft_max_procs should approach the same
value for a consistent workload and a machine not otherwise busy.
Depending on the needs of the system, these values may also diverge
if necessary to try to reach idle_target.
You may adjust these values if you wish by passing a value to the
method but you probably shouldn't. :)
initial_procs (read-only)
The number of processes to start running before attempting any
update_frequency
The minimum amount of time, in seconds, that must elapse between
checks of the system CPU's idle % and updates to the number of
running processes.
Set this to 0 to cause a check before each call to start().
Before each call to start() the time is compared with the last time a
check/update was performed. If this much time has passed, a new check
will be made of how busy the CPU is and the number of processes may
be adjusted.
default: 1
idle_target
Percentage of CPU idle time to try to maintain by adjusting the
number of running processes between hard_min_procs and hard_max_procs
default: 0 # try to keep the CPU 100% busy (0% idle)
idle_threshold
Only make adjustments if the current CPU idle % is this distance away
from idle_target. In other words, only adjust if abs(cur_idle -
idle_target) > idle_threshold. This may be a fractional value
(floating point).
You may notce that the default idle_target of 0 and idle_threshold of
1 would seem to indicate that the processes would never be adjusted
EXAMPLES
These examples are also provided in the examples/ directory of this
distribution.
Maximize CPU usage
see: examples/prun.pl
Run shell commands that are passed into the program and try to keep the
CPU busy, i.e. 0% idle
use Parallel::ForkManager::Scaled;
my $pm = Parallel::ForkManager::Scaled->new(
run_on_update => \&Parallel::ForkManager::Scaled::dump_stats
);
# just to be sure we can saturate the CPU
$pm->hard_max_procs($pm->ncpus * 4);
$pm->set_waitpid_blocking_sleep(0);
for my $i (0..1000) {
$pm->start and next;
my $start = time;
srand($$);
my $lifespan = 5+int(rand(10));
# Keep the CPU busy until it's time to exit
while (time - $start < $lifespan) {
my $a = time;
my $b = $a^time/3;
}
$pm->finish;
}
NOTES
examples/dummyload.pl view on Meta::CPAN
$pm->set_waitpid_blocking_sleep(0);
for my $i (0..1000) {
$pm->start and next;
my $start = time;
srand($$);
my $lifespan = 5+int(rand(10));
# Keep the CPU busy until it's time to exit
while (time - $start < $lifespan) {
my $a = time;
my $b = $a^time/3;
}
$pm->finish;
}
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
+ max(1, int(
($self->hard_max_procs - $self->max_procs)
* ($self->idle - $self->idle_target)
/ 100
))
)
);
}
#
# Decrease soft_min_procs, the system is too busy
#
sub adjust_soft_min {
my $self = shift;
$self->soft_min_procs(
max($self->hard_min_procs,
$self->hard_min_procs
+ max(0, int(
($self->max_procs - $self->hard_min_procs)
* ($self->idle_target - $self->idle)
/ 100
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
# do some work ..
# Exit the child
$pm->finish;
}
=head1 DESCRIPTION
This module inherits from Parallel::ForkManager and adds the ability
to automatically manage the number of processes running based on how
busy the system is by watching the CPU idle time. Each time a child is
about to be start()ed a new value for B<max_procs> may be calculated
(if enough time has passed since the last calculation). If a new value
is calculated, the number of processes to run will be adjusted by
calling B<set_max_procs> with the new value.
Without specifying any attributes to the constructor, some defaults will
be set for you (see Attributes below).
=head2 Attributes
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
=item B<soft_min_procs>
=item B<soft_max_procs>
This is initially set to B<hard_min_procs> and B<hard_max_procs> respectively
and is adjusted over time. These are used when calculating adjustments as the
minimum and maximum number of processes respectively.
Over time B<soft_min_procs> and B<soft_max_procs> should approach the same value
for a consistent workload and a machine not otherwise busy.
Depending on the needs of the system, these values may also diverge if
necessary to try to reach B<idle_target>.
You may adjust these values if you wish by passing a value to the method
but you probably shouldn't. :)
=item B<initial_procs> (read-only)
The number of processes to start running before attempting any adjustments,
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
=item B<update_frequency>
The minimum amount of time, in seconds, that must elapse between checks
of the system CPU's idle % and updates to the number of running processes.
Set this to 0 to cause a check before each call to C<start()>.
Before each call to C<start()> the time is compared with the last time a
check/update was performed. If this much time has passed, a new check will be
made of how busy the CPU is and the number of processes may be adjusted.
default: 1
=item B<idle_target>
Percentage of CPU idle time to try to maintain by adjusting the number of running
processes between B<hard_min_procs> and B<hard_max_procs>
default: 0 # try to keep the CPU 100% busy (0% idle)
=item B<idle_threshold>
Only make adjustments if the current CPU idle % is this distance away from B<idle_target>.
In other words, only adjust if C<abs(B<cur_idle> - B<idle_target>) E<gt> B<idle_threshold>>.
This may be a fractional value (floating point).
You may notce that the default B<idle_target> of 0 and B<idle_threshold> of 1
would seem to indicate that the processes would never be adjusted as idle can
never be less than 0%. At the limits, the threshold is adjusted so that we
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
=head1 EXAMPLES
These examples are also provided in the examples/ directory of
this distribution.
=head2 Maximize CPU usage
see: examples/prun.pl
Run shell commands that are passed into the program and try to
keep the CPU busy, i.e. 0% idle
use Parallel::ForkManager::Scaled;
my $pm = Parallel::ForkManager::Scaled->new(
run_on_update => \&Parallel::ForkManager::Scaled::dump_stats
);
# just to be sure we can saturate the CPU
$pm->hard_max_procs($pm->ncpus * 4);
lib/Parallel/ForkManager/Scaled.pm view on Meta::CPAN
$pm->set_waitpid_blocking_sleep(0);
for my $i (0..1000) {
$pm->start and next;
my $start = time;
srand($$);
my $lifespan = 5+int(rand(10));
# Keep the CPU busy until it's time to exit
while (time - $start < $lifespan) {
my $a = time;
my $b = $a^time/3;
}
$pm->finish;
}
=head1 NOTES
( run in 0.262 second using v1.01-cache-2.11-cpan-3cd7ad12f66 )