view release on metacpan or  search on metacpan

lib/Bencher/Backend.pm  view on Meta::CPAN

        'ScaleSize',
        ['RoundNumbers', {scientific_notation => $opts->{scientific_notation}}],
        ($envres->[3]{'func.module_startup'} ? ('ModuleStartup') : ()),
        ($envres->[3]{'func.code_startup'} ? ('CodeStartup') : ()),
        'DeleteConstantFields',
        'DeleteNotesFieldIfEmpty',
        'DeleteSeqField',

        ('RenderAsTextTable') x !!$opts->{render_as_text_table},
        ('RenderAsBenchmarkPm') x !!$opts->{render_as_benchmark_pm},
    ];

    # load all formatter modules
    my @fmtobjs;
    for my $fmt (@$formatters) {
        my ($fmtname, $fmtargs);
        if (ref($fmt)) {
            $fmtname = $fmt->[0];
            $fmtargs = $fmt->[1];
        } else {
            $fmtname = $fmt;
            $fmtargs = {};
        }
        my $fmtmod = "Bencher::Formatter::$fmtname";
        my $fmtmod_pm = $fmtmod; $fmtmod_pm =~ s!::!/!g; $fmtmod_pm .= ".pm";
        require $fmtmod_pm;
        push @fmtobjs, $fmtmod->new(%$fmtargs);
    }

    # run all munge_result()
    for my $fmtobj (@fmtobjs) {
        next unless $fmtobj->can("munge_result");
        $fmtobj->munge_result($envres);
    }

    # return the first render_result()
    for my $fmtobj (@fmtobjs) {
        next unless $fmtobj->can("render_result");
        return $fmtobj->render_result($envres);
    }

    # no render_result() has been called, we return the envres
    $envres;
}

# in enhanced mode, foo_bar becomes subscript etc. we don't want this. chart
# title can be non-enhanced with title => { text=>..., enhanced=>0 }, but
# dataset title can't. so we use escape stuffs.
sub _esc_gnuplot_title {
    my $val = shift;
    $val =~ s/_/-/g; # XXX superscript, greek?
    $val;
}

$SPEC{chart_result} = {
    v => 1.1,
    summary => 'Generate chart from the result',
    description => <<'_',

Will use gnuplot (via <pm:Chart::Gnuplot>) to generate the chart. Will produce
`.png` files in the specified directory.

Currently only results with one or two permutations of different items will be
chartable.

Options to customize the look/style of the chart will be added in the future.

_
    args => {
        envres => {
            summary => 'Enveloped result from bencher',
            schema => 'array*', # XXX envres
            req => 1,
            pos => 0,
        },
        output_file => {
            summary => '',
            schema => 'str*', # XXX filename
            req => 1,
            pos => 1,
            cmdline_aliases => {o=>{}},
            tags => ['category:output'],
        },
        overwrite => {
            schema => 'bool',
            tags => ['category:output'],
        },
        title => {
            schema => 'str*',
        },
    },
};
sub chart_result {
    require Chart::Gnuplot;

    my %args = @_;

    return [412, "Output file already exists, use overwrite=1 if you want to ".
                "overwrite the file"]
        if (-f $args{output_file}) && !$args{overwrite};
    return [412, "Result has no items, can't chart"] unless @{$args{envres}[2]};
    my $envres = format_result($args{envres}, undef, {render_as_text_table=>0});
    _set_item_names($envres->[2]);
    my $data = $envres->[3]{'func.module_startup'} || $envres->[3]{'func.code_startup'} ? "time" : "rate";

    my $chart = Chart::Gnuplot->new(
        #imagesize => "0.5, 0.5",
        output => $args{output_file},
        title  => $args{title} // 'Benchmark result'.($data eq 'rate' ? " (higher is better)" : "(shorter is better)"),
        ylabel => $data,
        xlabel => "",
        xtics  => {rotate=>"30 right"},
    );

    my (@ydata, @xdata);
    for my $it (@{ $envres->[2] }) {
        push @xdata, _esc_gnuplot_title($it->{_succinct_name});
        push @ydata, $it->{$data};
    }
    my @chart_datasets;
    my $ds = Chart::Gnuplot::DataSet->new(

lib/Bencher/Backend.pm  view on Meta::CPAN

Whether to test participant code once first before benchmarking.

By default, participant code is run once first for testing (e.g. whether it dies
or return the correct result) before benchmarking. If your code runs for many
seconds, you might want to skip this test and set this to 0.

=item * B<tidy> => I<bool>

Run perltidy over generated scripts.

=item * B<with_args_size> => I<bool>

Also return memory usage of item's arguments.

Memory size is measured using L<Devel::Size>.

=item * B<with_process_size> => I<bool>

Also return process size information for each item.

This is done by dumping each item's code into a temporary file and running the
file with a new perl interpreter process and measuring the process size at the
end (so it does not need to load Bencher itself or the other items). Currently
only works on Linux because process size information is retrieved from
C</proc/PID/smaps>. Not all code can work, e.g. if the code tries to access a
closure or outside data or extra modules (modules not specified in the
participant or loaded by the code itself). Usually does not make sense to use
this on external command participants.

=item * B<with_result_size> => I<bool>

Also return memory usage of each item code's result (return value).

Memory size is measured using L<Devel::Size>.


=back

Returns an enveloped result (an array).

First element ($status_code) is an integer containing HTTP-like status code
(200 means OK, 4xx caller error, 5xx function error). Second element
($reason) is a string containing error message, or something like "OK" if status is
200. Third element ($payload) is the actual result, but usually not present when enveloped result is an error response ($status_code is not 2xx). Fourth
element (%result_meta) is called result metadata and is optional, a hash
that contains extra information, much like how HTTP response headers provide additional metadata.

Return value:  (any)



=head2 chart_result

Usage:

 chart_result(%args) -> [$status_code, $reason, $payload, \%result_meta]

Generate chart from the result.

Will use gnuplot (via L<Chart::Gnuplot>) to generate the chart. Will produce
C<.png> files in the specified directory.

Currently only results with one or two permutations of different items will be
chartable.

Options to customize the look/style of the chart will be added in the future.

This function is not exported by default, but exportable.

Arguments ('*' denotes required arguments):

=over 4

=item * B<envres>* => I<array>

Enveloped result from bencher.

=item * B<output_file>* => I<str>

.

=item * B<overwrite> => I<bool>

(No description)

=item * B<title> => I<str>

(No description)


=back

Returns an enveloped result (an array).

First element ($status_code) is an integer containing HTTP-like status code
(200 means OK, 4xx caller error, 5xx function error). Second element
($reason) is a string containing error message, or something like "OK" if status is
200. Third element ($payload) is the actual result, but usually not present when enveloped result is an error response ($status_code is not 2xx). Fourth
element (%result_meta) is called result metadata and is optional, a hash
that contains extra information, much like how HTTP response headers provide additional metadata.

Return value:  (any)



=head2 format_result

Usage:

 format_result($envres, $formatters, $options, $exclude_formatters) -> [$status_code, $reason, $payload, \%result_meta]

Format bencher result.

This function is not exported by default, but exportable.

Arguments ('*' denotes required arguments):

=over 4

=item * B<$envres>* => I<array>