App-RecordStream
view release on metacpan or search on metacpan
4.0.25 2018-06-06 08:48:39 PDT
[Documentation]
* totable: Correct --delim option name in help. Thanks to Tomohiro
Hosaka for the report!
4.0.24 2017-07-31 09:41:23 PDT
[Tests]
* Remove duplicate key in JSON input that caused errors from
Cpanel::JSON::XS on CPAN Testers. Reported by Slaven ReziÄ as
[rt.cpan.org #122646].
4.0.23 2017-07-19 10:18:50 PDT
[Features]
* tojsonarray: A new command for outputting the record stream as a
single JSON array. Complements the existing fromjsonarray.
changes noted below, this release is largely the same as 4.0.10-TRIAL.
âtsibley
[Installation]
* Don't offer fromtcpdump on Perl < 5.10 due to NetPacket support
[Tests]
* Don't use bash-isms in tests, less /bin/sh is something like dash
* Travis CI integration: https://travis-ci.org/benbernard/RecordStream
4.0.10-TRIAL 2015-01-07 14:50:24 PST - TRIAL DEVELOPMENT RELEASE
[Documentation]
* Cleaned up App::RecordStream/README overview
* New installation instructions
* Documented the recs command and how to extend recs
* Removed outdated documentation for alternate install methods
* Added a "recs help" command
* Provided examples and story as "recs examples" and "recs story",
as well as linkable POD
* fromps: Document default fields for both Linux and OS X, regardless
of the OS of the releasing author
doc/recs-toptable.pod view on Meta::CPAN
Now lets see what happens when we have more than 1 left over field. Lets also
sum up the rss usage of the processes with -a sum,rss on recs-collate:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
{"priority":0,"count":4,"state":"sleep","sum_rss":471040}
{"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
{"priority":19,"count":1,"state":"sleep","sum_rss":0}
{"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
{"priority":0,"count":1,"state":"run","sum_rss":4784128}
{"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
{"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
{"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
{"priority":19,"count":2,"state":"run","sum_rss":8757248}
So now we have 2 left over fields that aren't columns, count and sum_rss. What
happens to our table now:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
+-----+--------+-------+-------+--------+
| |priority|0 |19 |ALL |
+-----+--------+-------+-------+--------+
|state| | | | |
+-----+--------+-------+-------+--------+
|ALL | |5255168|8757248|14012416|
+-----+--------+-------+-------+--------+
|run | |4784128|8757248|13541376|
+-----+--------+-------+-------+--------+
|sleep| |471040 |0 |471040 |
+-----+--------+-------+-------+--------+
We now have sum_rss values in this field. What if we want the other field
(count) displayed? We just use --v-field to specify what value field to use:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state --v count
doc/recs-toptable.pod view on Meta::CPAN
this by using the special FIELD specifier like so:
$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|ALL | |5 |5255168|3 |8757248|8 |14012416|
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
|sleep| |4 |471040 |1 |0 |5 |471040 |
+-----+--------+-----+-------+-----+-------+-----+--------+
So, now in one table we can see all the intersections of state and priority
values with the count and sum_rss fields. Remember that the ALL field (row and
column) are provided by the --cube functionality of recs-collate
lib/App/RecordStream/Manual/Story.pm view on Meta::CPAN
head -n 5 | recs-totable
Since JSON records are one to a line, we can use good ol' UNIX head to take the
5 top offenders. And we use recs-totable to convert those top ten to a nicely
formatted text table:
avg_latency ct url
----------------- ---- -----------------------
21.0909090909091 11 /dbfiles/list.cgi
1.36368901114811 6907 /view_image.cgi
1.02898550724638 345 /helpdesk.html
1 1 /dbfiles/
0.727272727272727 11 /linkGenerator/Host.cgi
=head1 And so it ends
When faced with awesome prowess like this, what can a 346-foot, 26000-ton sea
monster from beyond the stars do but slink back to its cave and bide its time
beneath downtown Seattle?
Should you find yourself locked in mortal combat with a unspeakable horror of
lib/App/RecordStream/Operation/toptable.pm view on Meta::CPAN
Now lets see what happens when we have more than 1 left over field. Lets also
sum up the rss usage of the processes with -a sum,rss on recs-collate:
__FORMAT_TEXT__
\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
{"priority":0,"count":4,"state":"sleep","sum_rss":471040}
{"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
{"priority":19,"count":1,"state":"sleep","sum_rss":0}
{"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
{"priority":0,"count":1,"state":"run","sum_rss":4784128}
{"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
{"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
{"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
{"priority":19,"count":2,"state":"run","sum_rss":8757248}
__FORMAT_TEXT__
So now we have 2 left over fields that aren't columns, count and sum_rss. What
happens to our table now:
__FORMAT_TEXT__
\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
+-----+--------+-------+-------+--------+
| |priority|0 |19 |ALL |
+-----+--------+-------+-------+--------+
|state| | | | |
+-----+--------+-------+-------+--------+
|ALL | |5255168|8757248|14012416|
+-----+--------+-------+-------+--------+
|run | |4784128|8757248|13541376|
+-----+--------+-------+-------+--------+
|sleep| |471040 |0 |471040 |
+-----+--------+-------+-------+--------+
__FORMAT_TEXT__
We now have sum_rss values in this field. What if we want the other field
(count) displayed? We just use --v-field to specify what value field to
use:
lib/App/RecordStream/Operation/toptable.pm view on Meta::CPAN
__FORMAT_TEXT__
\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
+-----+--------+-----+-------+-----+-------+-----+--------+
| |priority|0 | |19 | |ALL | |
+-----+--------+-----+-------+-----+-------+-----+--------+
| |FIELD |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state| | | | | | | |
+-----+--------+-----+-------+-----+-------+-----+--------+
|ALL | |5 |5255168|3 |8757248|8 |14012416|
+-----+--------+-----+-------+-----+-------+-----+--------+
|run | |1 |4784128|2 |8757248|3 |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
|sleep| |4 |471040 |1 |0 |5 |471040 |
+-----+--------+-----+-------+-----+-------+-----+--------+
__FORMAT_TEXT__
So, now in one table we can see all the intersections of state and priority
values with the count and sum_rss fields. Remember that the ALL field (row and
column) are provided by the --cube functionality of recs-collate
logos/logo-small.svg view on Meta::CPAN
rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
id="defs6" /><g
transform="matrix(1.25,0,0,-1.25,0,562.8)"
id="g10"><g
transform="scale(0.1,0.1)"
id="g12"><g
transform="scale(4682.4,4502.4)"
id="g14"><image
id="image16"
xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB58AAAdUCAYAAABwqM6aAAAABHNCSVQICAgIfAhkiAAAIABJREFUeJzs3enPZOeZ3/fvfZZan61XNimqKQ01oqCZ0YwRDGADiT0IkEQvjMR5GwTJ3+A/YBIBCeIEhpEJECAGnCAxEscxJh54RjOGI2k00mijdlGkhhqKokh2k713P2stZ7nvv...
transform="matrix(1,0,0,-1,0,1)"
preserveAspectRatio="none"
height="1"
width="1" /></g></g></g></svg>
logos/logo.svg view on Meta::CPAN
rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
id="defs6" /><g
transform="matrix(1.25,0,0,-1.25,0,562.8)"
id="g10"><g
transform="scale(0.1,0.1)"
id="g12"><g
transform="scale(4682.4,4502.4)"
id="g14"><image
id="image16"
xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB58AAAdUCAYAAABwqM6aAAAABHNCSVQICAgIfAhkiAAAIABJREFUeJzs3enPZOeZ3/fvfZZan61XNimqKQ01oqCZ0YwRDGADiT0IkEQvjMR5GwTJ3+A/YBIBCeIEhpEJECAGnCAxEscxJh54RjOGI2k00mijdlGkhhqKokh2k713P2stZ7nvv...
transform="matrix(1,0,0,-1,0,1)"
preserveAspectRatio="none"
height="1"
width="1" /></g></g></g></svg>
src/fast-recs-collate/json.h view on Meta::CPAN
char json_bc[] =
"\102\103\107\110\001\020\000\000\020\000\000\000\023\004\211\214\001\124\250\210"
"\010\100\105\106\001\102\144\204\034\040\104\210\314\004\201\042\043\200\311\231"
"\040\140\144\004\040\103\104\310\000\042\104\044\110\051\100\206\012\025\042\122"
"\016\220\241\102\205\210\210\005\144\204\044\040\143\104\000\000\051\020\000\000"
"\034\000\000\000\104\270\353\154\066\031\356\172\133\134\226\023\101\227\327\313"
"\124\230\303\354\171\231\202\055\005\135\012\164\244\334\321\151\172\171\016\017"
"\217\313\024\354\051\020\121\320\247\200\105\114\335\235\036\277\311\345\357\030"
"\035\226\223\043\016\217\327\163\166\170\216\376\216\321\141\071\025\143\164\130"
"\076\047\143\116\227\247\335\147\102\346\164\371\327\054\017\237\251\260\207\331"
"\365\062\031\157\261\272\074\246\123\021\226\313\303\171\000\000\041\020\000\000"
"\106\000\000\000\221\100\000\000\050\000\000\000\005\003\001\100\043\020\020\004"
"\106\005\002\244\102\005\002\205\110\250\120\201\200\241\161\150\204\012\025\050"
"\204\012\004\120\005\002\211\200\252\121\241\004\201\125\241\302\045\050\070\326"
"\340\030\210\143\042\226\261\154\326\242\031\214\160\061\071\212\351\064\266\145"
"\130\127\142\146\221\271\101\106\327\331\036\145\175\227\271\054\126\204\131\031"
"\146\145\204\315\025\306\026\230\313\162\060\071\212\025\141\126\206\035\114\316"
"\143\154\307\045\050\070\326\340\030\210\163\060\071\217\025\141\126\206\131\335"
"\144\141\314\301\344\110\306\206\130\035\147\053\231\265\144\007\223\043\035\114"
"\216\144\060\302\305\344\050\126\066\230\134\005\221\100\000\000\031\000\000\000"
"\125\010\044\002\214\100\253\121\203\206\053\124\243\306\001\204\030\046\022\334"
"\310\226\141\134\307\165\375\007\030\007\223\223\134\220\121\162\141\146\311\301"
"\344\040\027\144\204\134\230\031\162\000\041\306\215\154\031\327\365\037\140\230"
( run in 0.678 second using v1.01-cache-2.11-cpan-7add2cbd662 )