Forks-Super
view release on metacpan or search on metacpan
examples/Benchmark.pm view on Meta::CPAN
'Name2' => sub { ...code2... },
},
'none'
);
cmpthese( $results ) ;
$t = timeit($count, '...other code...')
print "$count loops of other code took:",timestr($t),"\n";
$t = countit($time, '...other code...')
$count = $t->iters ;
print "$count loops of other code took:",timestr($t),"\n";
# enable hires wallclock timing if possible
use Benchmark ':hireswallclock';
# execute code in parallel if possible
use Benchmark ':parallel';
=head1 DESCRIPTION
The Benchmark module encapsulates a number of routines to help you
figure out how long it takes to execute some code.
timethis - run a chunk of code several times
timethese - run several chunks of code several times
cmpthese - print results of timethese as a comparison chart
timeit - run a chunk of code and see how long it goes
countit - see how many times a chunk of code runs in a given time
=head2 Methods
=over 10
=item new
Returns the current time. Example:
use Benchmark;
$t0 = Benchmark->new;
# ... your code here ...
$t1 = Benchmark->new;
$td = timediff($t1, $t0);
print "the code took:",timestr($td),"\n";
=item debug
Enables or disable debugging by setting the C<$Benchmark::Debug> flag:
Benchmark->debug(1);
$t = timeit(10, ' 5 ** $Global ');
Benchmark->debug(0);
=item iters
Returns the number of iterations.
=back
=head2 Standard Exports
The following routines will be exported into your namespace
if you use the Benchmark module:
=over 10
=item timeit(COUNT, CODE)
Arguments: COUNT is the number of times to run the loop, and CODE is
the code to run. CODE may be either a code reference or a string to
be eval'd; either way it will be run in the caller's package.
Returns: a Benchmark object.
=item timethis ( COUNT, CODE, [ TITLE, [ STYLE ]] )
Time COUNT iterations of CODE. CODE may be a string to eval or a
code reference; either way the CODE will run in the caller's package.
Results will be printed to STDOUT as TITLE followed by the times.
TITLE defaults to "timethis COUNT" if none is provided. STYLE
determines the format of the output, as described for timestr() below.
The COUNT can be zero or negative: this means the I<minimum number of
CPU seconds> to run. A zero signifies the default of 3 seconds. For
example to run at least for 10 seconds:
timethis(-10, $code)
or to run two pieces of code tests for at least 3 seconds:
timethese(0, { test1 => '...', test2 => '...'})
CPU seconds is, in UNIX terms, the user time plus the system time of
the process itself, as opposed to the real (wallclock) time and the
time spent by the child processes. Less than 0.1 seconds is not
accepted (-0.01 as the count, for example, will cause a fatal runtime
exception).
Note that the CPU seconds is the B<minimum> time: CPU scheduling and
other operating system factors may complicate the attempt so that a
little bit more time is spent. The benchmark output will, however,
also tell the number of C<$code> runs/second, which should be a more
interesting number than the actually spent seconds.
Returns a Benchmark object.
=item timethese ( COUNT, CODEHASHREF, [ STYLE ] )
The CODEHASHREF is a reference to a hash containing names as keys
and either a string to eval or a code reference for each value.
For each (KEY, VALUE) pair in the CODEHASHREF, this routine will
call
timethis(COUNT, VALUE, KEY, STYLE)
The routines are called in string comparison order of KEY.
The COUNT can be zero or negative, see timethis().
Returns a hash reference of Benchmark objects, keyed by name.
=item timediff ( T1, T2 )
Returns the difference between two Benchmark times as a Benchmark
object suitable for passing to timestr().
=item timestr ( TIMEDIFF, [ STYLE, [ FORMAT ] ] )
Returns a string that formats the times in the TIMEDIFF object in
the requested STYLE. TIMEDIFF is expected to be a Benchmark object
similar to that returned by timediff().
STYLE can be any of 'all', 'none', 'noc', 'nop' or 'auto'. 'all' shows
each of the 5 times available ('wallclock' time, user time, system time,
user time of children, and system time of children). 'noc' shows all
except the two children times. 'nop' shows only wallclock and the
two children times. 'auto' (the default) will act as 'all' unless
examples/Benchmark.pm view on Meta::CPAN
The following routines will be exported into your namespace
if you specifically ask that they be imported:
=over 10
=item clearcache ( COUNT )
Clear the cached time for COUNT rounds of the null loop.
=item clearallcache ( )
Clear all cached times.
=item cmpthese ( COUNT, CODEHASHREF, [ STYLE ] )
=item cmpthese ( RESULTSHASHREF, [ STYLE ] )
Optionally calls timethese(), then outputs comparison chart. This:
cmpthese( -1, { a => "++\$i", b => "\$i *= 2" } ) ;
outputs a chart like:
Rate b a
b 2831802/s -- -61%
a 7208959/s 155% --
This chart is sorted from slowest to fastest, and shows the percent speed
difference between each pair of tests.
C<cmpthese> can also be passed the data structure that timethese() returns:
$results = timethese( -1,
{ a => "++\$i", b => "\$i *= 2" } ) ;
cmpthese( $results );
in case you want to see both sets of results.
If the first argument is an unblessed hash reference,
that is RESULTSHASHREF; otherwise that is COUNT.
Returns a reference to an ARRAY of rows, each row is an ARRAY of cells from the
above chart, including labels. This:
my $rows = cmpthese( -1,
{ a => '++$i', b => '$i *= 2' }, "none" );
returns a data structure like:
[
[ '', 'Rate', 'b', 'a' ],
[ 'b', '2885232/s', '--', '-59%' ],
[ 'a', '7099126/s', '146%', '--' ],
]
B<NOTE>: This result value differs from previous versions, which returned
the C<timethese()> result structure. If you want that, just use the two
statement C<timethese>...C<cmpthese> idiom shown above.
Incidentally, note the variance in the result values between the two examples;
this is typical of benchmarking. If this were a real benchmark, you would
probably want to run a lot more iterations.
=item countit(TIME, CODE)
Arguments: TIME is the minimum length of time to run CODE for, and CODE is
the code to run. CODE may be either a code reference or a string to
be eval'd; either way it will be run in the caller's package.
TIME is I<not> negative. countit() will run the loop many times to
calculate the speed of CODE before running it for TIME. The actual
time run for will usually be greater than TIME due to system clock
resolution, so it's best to look at the number of iterations divided
by the times that you are concerned with, not just the iterations.
Returns: a Benchmark object.
=item disablecache ( )
Disable caching of timings for the null loop. This will force Benchmark
to recalculate these timings for each new piece of code timed.
=item enablecache ( )
Enable caching of timings for the null loop. The time taken for COUNT
rounds of the null loop will be calculated only once for each
different COUNT used.
=item timesum ( T1, T2 )
Returns the sum of two Benchmark times as a Benchmark object suitable
for passing to timestr().
=back
=head2 :hireswallclock
If the Time::HiRes module has been installed, you can specify the
special tag C<:hireswallclock> for Benchmark (if Time::HiRes is not
available, the tag will be silently ignored). This tag will cause the
wallclock time to be measured in microseconds, instead of integer
seconds. Note though that the speed computations are still conducted
in CPU time, not wallclock time.
=head2 :parallel
If the L<Forks::Super> module has been installed, you can specify the
special tag C<:parallel> for Benchmark. This tag will cause the code
loops to be executed in parallel, so the benchmarking will take less
time. If L<Forks::Super> is not available, the tag will be ignored.
=head1 Benchmark Object
Many of the functions in this module return a Benchmark object,
or in the case of C<timethese()>, a reference to a hash, the values of
which are Benchmark objects. This is useful if you want to store or
further process results from Benchmark functions.
Internally the Benchmark object holds timing values,
described in L</"NOTES"> below.
The following methods can be used to access them:
=over 4
=item cpu_p
Total CPU (User + System) of the main (parent) process.
=item cpu_c
Total CPU (User + System) of any children processes.
=item cpu_a
Total CPU of parent and any children processes.
=item real
Real elapsed time "wallclock seconds".
=item iters
Number of iterations run.
=back
The following illustrates use of the Benchmark object:
$result = timethis(100000, sub { ... });
print "total CPU = ", $result->cpu_a, "\n";
=head1 NOTES
The data is stored as a list of values from the time and times
functions:
($real, $user, $system, $children_user, $children_system, $iters)
in seconds for the whole loop (not divided by the number of rounds).
The timing is done using time(3) and times(3).
Code is executed in the caller's package.
The time of the null loop (a loop with the same
number of rounds but empty loop body) is subtracted
from the time of the real loop.
The null loop times can be cached, the key being the
number of rounds. The caching can be controlled using
calls like these:
clearcache($key);
clearallcache();
disablecache();
enablecache();
Caching is off by default, as it can (usually slightly) decrease
accuracy and does not usually noticeably affect runtimes.
=head1 EXAMPLES
For example,
use Benchmark qw( cmpthese ) ;
$x = 3;
cmpthese( -5, {
a => sub{$x*$x},
b => sub{$x**2},
} );
outputs something like this:
Benchmark: running a, b, each for at least 5 CPU seconds...
Rate b a
b 1559428/s -- -62%
a 4152037/s 166% --
while
use Benchmark qw( timethese cmpthese ) ;
examples/Benchmark.pm view on Meta::CPAN
while ( ( $t0 = Benchmark->new(0) )->[1] == $tbase ) {
for (my $i=0; $i < $limit; $i++) { my $x = $i / 1.5 } # burn user CPU
$limit *= 1.1;
}
$subref->($_);
$t1 = Benchmark->new($_);
&timediff($t1, $t0);
} map {
$n -= my $nreps = int($n / $_);
$nreps;
} reverse(1 .. $nchild);
$td = shift @td;
foreach my $tdc (@td) {
$td->[$_] += $tdc->[$_] for 0 .. $#$tdc;
}
} else {
# Wait for the user timer to tick. This makes the error range more like
# -0.01, +0. If we don't wait, then it's more like -0.01, +0.01. This
# may not seem important, but it significantly reduces the chances of
# getting a too low initial $n in the initial, 'find the minimum' loop
# in &countit. This, in turn, can reduce the number of calls to
# &runloop a lot, and thus reduce additive errors.
#
# Note that its possible for the act of reading the system clock to
# burn lots of system CPU while we burn very little user clock in the
# busy loop, which can cause the loop to run for a very long wall time.
# So gradually ramp up the duration of the loop. See RT #122003
#
my $tbase = Benchmark->new(0)->[1];
my $limit = 1;
while ( ( $t0 = Benchmark->new(0) )->[1] == $tbase ) {
for (my $i=0; $i < $limit; $i++) { my $x = $i / 1.5 } # burn user CPU
$limit *= 1.1;
}
$subref->($n);
$t1 = Benchmark->new($n);
$td = &timediff($t1, $t0);
}
timedebug("runloop:",$td);
$td;
}
$_Usage{timeit} = <<'USAGE';
usage: $result = timeit($count, 'code' ); or
$result = timeit($count, sub { code } );
USAGE
sub timeit {
my($n, $code) = @_;
my($wn, $wc, $wd);
die usage unless defined $code and
(!ref $code or ref $code eq 'CODE');
printf STDERR "timeit $n $code\n" if $Debug;
my $cache_key = $n . ( ref( $code ) ? 'c' : 's' );
if ($Do_Cache && exists $Cache{$cache_key} ) {
$wn = $Cache{$cache_key};
} else {
$wn = &runloop($n, ref( $code ) ? sub { } : '' );
# Can't let our baseline have any iterations, or they get subtracted
# out of the result.
$wn->[5] = 0;
$Cache{$cache_key} = $wn;
}
$wc = &runloop($n, $code);
$wd = timediff($wc, $wn);
timedebug("timeit: ",$wc);
timedebug(" - ",$wn);
timedebug(" = ",$wd);
$wd;
}
my $default_for = 3;
my $min_for = 0.1;
$_Usage{countit} = <<'USAGE';
usage: $result = countit($time, 'code' ); or
$result = countit($time, sub { code } );
USAGE
sub countit {
my ( $tmax, $code ) = @_;
die usage unless @_;
if ( not defined $tmax or $tmax == 0 ) {
$tmax = $default_for;
} elsif ( $tmax < 0 ) {
$tmax = -$tmax;
}
die "countit($tmax, ...): timelimit cannot be less than $min_for.\n"
if $tmax < $min_for;
my ($n, $tc);
# First find the minimum $n that gives a significant timing.
my $zeros=0;
for ($n = 1; ; $n *= 2 ) {
my $t0 = Benchmark->new(0);
my $td = timeit($n, $code);
my $t1 = Benchmark->new(0);
$tc = $td->[1] + $td->[2];
if ( $tc <= 0 and $n > 1024 ) {
my $d = timediff($t1, $t0);
# note that $d is the total CPU time taken to call timeit(),
# while $tc is is difference in CPU secs between the empty run
# and the code run. If the code is trivial, its possible
# for $d to get large while $tc is still zero (or slightly
# negative). Bail out once timeit() starts taking more than a
# few seconds without noticeable difference.
if ($d->[1] + $d->[2] > 8
|| ++$zeros > 16)
{
die "Timing is consistently zero in estimation loop, cannot benchmark. N=$n\n";
examples/Benchmark.pm view on Meta::CPAN
$cstot += $td->[4];
$ttot = $utot + $stot;
last if $ttot >= $tmax;
if ( $ttot <= 0 ) {
++$zeros > 16
and die "Timing is consistently zero, cannot benchmark. N=$n\n";
} else {
$zeros = 0;
}
$ttot = 0.01 if $ttot < 0.01;
my $r = $tmax / $ttot - 1; # Linear approximation.
$n = int( $r * $ntot );
$n = $nmin if $n < $nmin;
}
return bless [ $rtot, $utot, $stot, $cutot, $cstot, $ntot ];
}
# --- Functions implementing high-level time-then-print utilities
sub n_to_for {
my $n = shift;
return $n == 0 ? $default_for : $n < 0 ? -$n : undef;
}
$_Usage{timethis} = <<'USAGE';
usage: $result = timethis($time, 'code' ); or
$result = timethis($time, sub { code } );
USAGE
sub timethis{
my($n, $code, $title, $style) = @_;
my($t, $forn);
die usage unless defined $code and
(!ref $code or ref $code eq 'CODE');
if ( $n > 0 ) {
croak "non-integer loopcount $n, stopped" if int($n)<$n;
$t = timeit($n, $code);
$title = "timethis $n" unless defined $title;
} else {
my $fort = n_to_for( $n );
$t = countit( $fort, $code );
$title = "timethis for $fort" unless defined $title;
$forn = $t->[-1];
}
local $| = 1;
$style = "" unless defined $style;
printf("%10s: ", $title) unless $style eq 'none';
print timestr($t, $style, $Default_Format),"\n" unless $style eq 'none';
$n = $forn if defined $forn;
if ($t->elapsed($style) < 0) {
# due to clock granularity and variable CPU speed and load,
# on quick code with a small number of loops, it's possible for
# the empty loop to appear to take longer than the real loop
# (e.g. 1 tick versus 0 ticks). This leads to a negative elapsed
# time. In this case, floor it at zero, to stop bizarre results.
print " (warning: too few iterations for a reliable count)\n";
$t->[$_] = 0 for 1..4;
}
# A conservative warning to spot very silly tests.
# Don't assume that your benchmark is ok simply because
# you don't get this warning!
print " (warning: too few iterations for a reliable count)\n"
if $n < $Min_Count
|| ($t->real < 1 && $n < 1000)
|| $t->cpu_a < $Min_CPU;
$t;
}
$_Usage{timethese} = <<'USAGE';
usage: timethese($count, { Name1 => 'code1', ... }); or
timethese($count, { Name1 => sub { code1 }, ... });
USAGE
sub timethese{
my($n, $alt, $style) = @_;
die usage unless ref $alt eq 'HASH';
my @names = sort keys %$alt;
$style = "" unless defined $style;
print "Benchmark: " unless $style eq 'none';
if ( $n > 0 ) {
croak "non-integer loopcount $n, stopped" if int($n)<$n;
print "timing $n iterations of" unless $style eq 'none';
} else {
print "running" unless $style eq 'none';
}
print " ", join(', ',@names) unless $style eq 'none';
unless ( $n > 0 ) {
my $for = n_to_for( $n );
print ", each" if $n > 1 && $style ne 'none';
print " for at least $for CPU seconds" unless $style eq 'none';
}
print "...\n" unless $style eq 'none';
# we could save the results in an array and produce a summary here
# sum, min, max, avg etc etc
my %results;
foreach my $name (@names) {
$results{$name} = timethis ($n, $alt -> {$name}, $name, $style);
}
return \%results;
}
$_Usage{cmpthese} = <<'USAGE';
usage: cmpthese($count, { Name1 => 'code1', ... }); or
cmpthese($count, { Name1 => sub { code1 }, ... }); or
cmpthese($result, $style);
USAGE
sub cmpthese{
my ($results, $style);
# $count can be a blessed object.
if ( ref $_[0] eq 'HASH' ) {
($results, $style) = @_;
}
else {
my($count, $code) = @_[0,1];
$style = $_[2] if defined $_[2];
die usage unless ref $code eq 'HASH';
$results = timethese($count, $code, ($style || "none"));
}
$style = "" unless defined $style;
# Flatten in to an array of arrays with the name as the first field
my @vals = map{ [ $_, @{$results->{$_}} ] } keys %$results;
for (@vals) {
# recreate the pre-flattened Benchmark object
my $tmp_bm = bless [ @{$_}[1..$#$_] ];
my $elapsed = $tmp_bm->elapsed($style);
# The epsilon fudge here is to prevent div by 0. Since clock
# resolutions are much larger, it's below the noise floor.
my $rate = $_->[6]/(($elapsed)+0.000000000000001);
$_->[7] = $rate;
}
# Sort by rate
( run in 1.411 second using v1.01-cache-2.11-cpan-71847e10f99 )