view release on metacpan or  search on metacpan

lib/App/RecordStream/Manual/Story.pm  view on Meta::CPAN

this case we are using a substitution command to strip the key off of get.cgi
requests. At this point our data is ready to be aggregated and made into
statistics.

=head1 recs-collate - Generate aggregate statistics

recs-collate -k url --perfect -a avg,latency -a count

recs-collate is the crown jewel of recs analysis. It groups records from input
together, computes aggregate information about them, and dumps this aggregate
information as output records. "-k url" requests that records be grouped by
their "url" field. "--perfect" indicates that they should be grouped together
even if they are not adjacent in input (adjacent only is the default). "-a
avg,latency" requests that the average aggregator be used on the latency field.
"-a count" requests that the count aggregator be used.

Aggregators are one of the most powerful features of recs. As of writing there
are 21 distinct aggregators ready for use. Some of the most powerful are:

  average: averages provided field
  count: counts (non-unique) records
  distinctcount: count unique values from provided field
  maximum: maximum value for a field
  percentile: value of pXX for field
  sum: sums provided field

You can find out what all of them are with `recs-collate --list-aggregators`.

Here are a few sample records from after the collate step:

  {"count":11,"url":"/dbfiles/list.cgi","avg_latency":21.0909090909091}
  {"count":2,"url":"/linkGenerator/Host.cgi","avg_latency":0.5}
  {"count":3,"url":"/view_image.cgi","avg_latency":0.333333333333333}
  {"count":21,"url":"/dbfiles/check.cgi","avg_latency":0.476190476190476}

=head1 recs-sort - ordering records in a stream

Now that the collation has been done the records have the numbers we desire, but they are neither in a useful order, nor a pretty format.

The first we rectify with recs-sort:

recs-sort -k 'avg_latency=-n'

We have specified that the records are to be sorted by their avg_latency field
and they are to be sorted numerically, descending (negative n)

=head1 recs-totable - pretty output of data

Finally, we convert JSON back to something slightly more human readable:

head -n 5 | recs-totable

Since JSON records are one to a line, we can use good ol' UNIX head to take the
5 top offenders. And we use recs-totable to convert those top ten to a nicely
formatted text table:

  avg_latency         ct     url
  -----------------   ----   -----------------------
  21.0909090909091    11     /dbfiles/list.cgi
  1.36368901114811    6907   /view_image.cgi
  1.02898550724638    345    /helpdesk.html
  1                   1      /dbfiles/
  0.727272727272727   11     /linkGenerator/Host.cgi

=head1 And so it ends

When faced with awesome prowess like this, what can a 346-foot, 26000-ton sea
monster from beyond the stars do but slink back to its cave and bide its time
beneath downtown Seattle?

Should you find yourself locked in mortal combat with a unspeakable horror of
your own you can always turn to --help. All recs scripts come equipped with
detailed usage instructions triggered by the --help option. You can also turn
to `man recs` (if the man file is installed correctly)

=head1 SEE ALSO

=over

=item * See L<App::RecordStream> for an overview of the scripts and the system

=item * Run C<recs examples> or see L<App::RecordStream::Manual::Examples> for a set of simple recs examples

=back

=cut

1;