App-RecordStream

 view release on metacpan or  search on metacpan

Changes  view on Meta::CPAN

4.0.25  2018-06-06 08:48:39 PDT
    [Documentation]
        * totable: Correct --delim option name in help.  Thanks to Tomohiro
          Hosaka for the report!

4.0.24  2017-07-31 09:41:23 PDT
    [Tests]
        * Remove duplicate key in JSON input that caused errors from
          Cpanel::JSON::XS on CPAN Testers.  Reported by Slaven Rezić as
          [rt.cpan.org #122646].

4.0.23  2017-07-19 10:18:50 PDT
    [Features]
        * tojsonarray: A new command for outputting the record stream as a
          single JSON array.  Complements the existing fromjsonarray.

Changes  view on Meta::CPAN

    changes noted below, this release is largely the same as 4.0.10-TRIAL.
        –tsibley

    [Installation]
        * Don't offer fromtcpdump on Perl < 5.10 due to NetPacket support

    [Tests]
        * Don't use bash-isms in tests, less /bin/sh is something like dash
        * Travis CI integration: https://travis-ci.org/benbernard/RecordStream

4.0.10-TRIAL  2015-01-07 14:50:24 PST - TRIAL DEVELOPMENT RELEASE
    [Documentation]
        * Cleaned up App::RecordStream/README overview
        * New installation instructions
        * Documented the recs command and how to extend recs
        * Removed outdated documentation for alternate install methods
        * Added a "recs help" command
        * Provided examples and story as "recs examples" and "recs story",
          as well as linkable POD
        * fromps: Document default fields for both Linux and OS X, regardless
          of the OS of the releasing author

doc/recs-toptable.pod  view on Meta::CPAN

 
 Now lets see what happens when we have more than 1 left over field. Lets also
 sum up the rss usage of the processes with -a sum,rss on recs-collate:
 
 $ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
 {"priority":0,"count":4,"state":"sleep","sum_rss":471040}
 {"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
 {"priority":19,"count":1,"state":"sleep","sum_rss":0}
 {"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
 {"priority":0,"count":1,"state":"run","sum_rss":4784128}
 {"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
 {"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
 {"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
 {"priority":19,"count":2,"state":"run","sum_rss":8757248}
 
 So now we have 2 left over fields that aren't columns, count and sum_rss. What
 happens to our table now:
 
 $ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
 +-----+--------+-------+-------+--------+
 |     |priority|0      |19     |ALL     |
 +-----+--------+-------+-------+--------+
 |state|        |       |       |        |
 +-----+--------+-------+-------+--------+
 |ALL  |        |5255168|8757248|14012416|
 +-----+--------+-------+-------+--------+
 |run  |        |4784128|8757248|13541376|
 +-----+--------+-------+-------+--------+
 |sleep|        |471040 |0      |471040  |
 +-----+--------+-------+-------+--------+
 
 We now have sum_rss values in this field. What if we want the other field
 (count) displayed? We just use --v-field to specify what value field to use:
 
 $ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state --v count

doc/recs-toptable.pod  view on Meta::CPAN

 this by using the special FIELD specifier like so:
 
 $ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |     |priority|0    |       |19   |       |ALL  |        |
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |     |FIELD   |count|sum_rss|count|sum_rss|count|sum_rss |
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |state|        |     |       |     |       |     |        |
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |ALL  |        |5    |5255168|3    |8757248|8    |14012416|
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |run  |        |1    |4784128|2    |8757248|3    |13541376|
 +-----+--------+-----+-------+-----+-------+-----+--------+
 |sleep|        |4    |471040 |1    |0      |5    |471040  |
 +-----+--------+-----+-------+-----+-------+-----+--------+
 
 So, now in one table we can see all the intersections of state and priority
 values with the count and sum_rss fields. Remember that the ALL field (row and
 column) are provided by the --cube functionality of recs-collate
 

lib/App/RecordStream/Manual/Story.pm  view on Meta::CPAN

head -n 5 | recs-totable

Since JSON records are one to a line, we can use good ol' UNIX head to take the
5 top offenders. And we use recs-totable to convert those top ten to a nicely
formatted text table:

  avg_latency         ct     url
  -----------------   ----   -----------------------
  21.0909090909091    11     /dbfiles/list.cgi
  1.36368901114811    6907   /view_image.cgi
  1.02898550724638    345    /helpdesk.html
  1                   1      /dbfiles/
  0.727272727272727   11     /linkGenerator/Host.cgi

=head1 And so it ends

When faced with awesome prowess like this, what can a 346-foot, 26000-ton sea
monster from beyond the stars do but slink back to its cave and bide its time
beneath downtown Seattle?

Should you find yourself locked in mortal combat with a unspeakable horror of

lib/App/RecordStream/Operation/toptable.pm  view on Meta::CPAN

Now lets see what happens when we have more than 1 left over field.  Lets also
sum up the rss usage of the processes with -a sum,rss on recs-collate:
__FORMAT_TEXT__

\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss
{"priority":0,"count":4,"state":"sleep","sum_rss":471040}
{"priority":"ALL","count":5,"state":"sleep","sum_rss":471040}
{"priority":19,"count":1,"state":"sleep","sum_rss":0}
{"priority":0,"count":5,"state":"ALL","sum_rss":5255168}
{"priority":0,"count":1,"state":"run","sum_rss":4784128}
{"priority":"ALL","count":8,"state":"ALL","sum_rss":14012416}
{"priority":"ALL","count":3,"state":"run","sum_rss":13541376}
{"priority":19,"count":3,"state":"ALL","sum_rss":8757248}
{"priority":19,"count":2,"state":"run","sum_rss":8757248}

__FORMAT_TEXT__
So now we have 2 left over fields that aren't columns, count and sum_rss.  What
happens to our table now:
__FORMAT_TEXT__

\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority --y state
+-----+--------+-------+-------+--------+
|     |priority|0      |19     |ALL     |
+-----+--------+-------+-------+--------+
|state|        |       |       |        |
+-----+--------+-------+-------+--------+
|ALL  |        |5255168|8757248|14012416|
+-----+--------+-------+-------+--------+
|run  |        |4784128|8757248|13541376|
+-----+--------+-------+-------+--------+
|sleep|        |471040 |0      |471040  |
+-----+--------+-------+-------+--------+

__FORMAT_TEXT__
We now have sum_rss values in this field.  What if we want the other field
(count) displayed?  We just use --v-field to specify what value field to
use:

lib/App/RecordStream/Operation/toptable.pm  view on Meta::CPAN

__FORMAT_TEXT__

\$ cat /var/tmp/psrecs | recs-collate --perfect --key priority,state -a count --cube -a sum,rss | recs-toptable --x priority,FIELD --y state
+-----+--------+-----+-------+-----+-------+-----+--------+
|     |priority|0    |       |19   |       |ALL  |        |
+-----+--------+-----+-------+-----+-------+-----+--------+
|     |FIELD   |count|sum_rss|count|sum_rss|count|sum_rss |
+-----+--------+-----+-------+-----+-------+-----+--------+
|state|        |     |       |     |       |     |        |
+-----+--------+-----+-------+-----+-------+-----+--------+
|ALL  |        |5    |5255168|3    |8757248|8    |14012416|
+-----+--------+-----+-------+-----+-------+-----+--------+
|run  |        |1    |4784128|2    |8757248|3    |13541376|
+-----+--------+-----+-------+-----+-------+-----+--------+
|sleep|        |4    |471040 |1    |0      |5    |471040  |
+-----+--------+-----+-------+-----+-------+-----+--------+

__FORMAT_TEXT__
So, now in one table we can see all the intersections of state and priority
values with the count and sum_rss fields.  Remember that the ALL field (row and
column) are provided by the --cube functionality of recs-collate

logos/logo-small.svg  view on Meta::CPAN

         rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
           rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
     id="defs6" /><g
     transform="matrix(1.25,0,0,-1.25,0,562.8)"
     id="g10"><g
       transform="scale(0.1,0.1)"
       id="g12"><g
         transform="scale(4682.4,4502.4)"
         id="g14"><image
           id="image16"
           xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB58AAAdUCAYAAABwqM6aAAAABHNCSVQICAgIfAhkiAAAIABJREFUeJzs3enPZOeZ3/fvfZZan61XNimqKQ01oqCZ0YwRDGADiT0IkEQvjMR5GwTJ3+A/YBIBCeIEhpEJECAGnCAxEscxJh54RjOGI2k00mijdlGkhhqKokh2k713P2stZ7nvv...
           transform="matrix(1,0,0,-1,0,1)"
           preserveAspectRatio="none"
           height="1"
           width="1" /></g></g></g></svg>

logos/logo.svg  view on Meta::CPAN

         rdf:about=""><dc:format>image/svg+xml</dc:format><dc:type
           rdf:resource="http://purl.org/dc/dcmitype/StillImage" /></cc:Work></rdf:RDF></metadata><defs
     id="defs6" /><g
     transform="matrix(1.25,0,0,-1.25,0,562.8)"
     id="g10"><g
       transform="scale(0.1,0.1)"
       id="g12"><g
         transform="scale(4682.4,4502.4)"
         id="g14"><image
           id="image16"
           xlink:href="data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAB58AAAdUCAYAAABwqM6aAAAABHNCSVQICAgIfAhkiAAAIABJREFUeJzs3enPZOeZ3/fvfZZan61XNimqKQ01oqCZ0YwRDGADiT0IkEQvjMR5GwTJ3+A/YBIBCeIEhpEJECAGnCAxEscxJh54RjOGI2k00mijdlGkhhqKokh2k713P2stZ7nvv...
           transform="matrix(1,0,0,-1,0,1)"
           preserveAspectRatio="none"
           height="1"
           width="1" /></g></g></g></svg>

src/fast-recs-collate/json.h  view on Meta::CPAN


char json_bc[] =
"\102\103\107\110\001\020\000\000\020\000\000\000\023\004\211\214\001\124\250\210"
"\010\100\105\106\001\102\144\204\034\040\104\210\314\004\201\042\043\200\311\231"
"\040\140\144\004\040\103\104\310\000\042\104\044\110\051\100\206\012\025\042\122"
"\016\220\241\102\205\210\210\005\144\204\044\040\143\104\000\000\051\020\000\000"
"\034\000\000\000\104\270\353\154\066\031\356\172\133\134\226\023\101\227\327\313"
"\124\230\303\354\171\231\202\055\005\135\012\164\244\334\321\151\172\171\016\017"
"\217\313\024\354\051\020\121\320\247\200\105\114\335\235\036\277\311\345\357\030"
"\035\226\223\043\016\217\327\163\166\170\216\376\216\321\141\071\025\143\164\130"
"\076\047\143\116\227\247\335\147\102\346\164\371\327\054\017\237\251\260\207\331"
"\365\062\031\157\261\272\074\246\123\021\226\313\303\171\000\000\041\020\000\000"
"\106\000\000\000\221\100\000\000\050\000\000\000\005\003\001\100\043\020\020\004"
"\106\005\002\244\102\005\002\205\110\250\120\201\200\241\161\150\204\012\025\050"
"\204\012\004\120\005\002\211\200\252\121\241\004\201\125\241\302\045\050\070\326"
"\340\030\210\143\042\226\261\154\326\242\031\214\160\061\071\212\351\064\266\145"
"\130\127\142\146\221\271\101\106\327\331\036\145\175\227\271\054\126\204\131\031"
"\146\145\204\315\025\306\026\230\313\162\060\071\212\025\141\126\206\035\114\316"
"\143\154\307\045\050\070\326\340\030\210\163\060\071\217\025\141\126\206\131\335"
"\144\141\314\301\344\110\306\206\130\035\147\053\231\265\144\007\223\043\035\114"
"\216\144\060\302\305\344\050\126\066\230\134\005\221\100\000\000\031\000\000\000"
"\125\010\044\002\214\100\253\121\203\206\053\124\243\306\001\204\030\046\022\334"
"\310\226\141\134\307\165\375\007\030\007\223\223\134\220\121\162\141\146\311\301"
"\344\040\027\144\204\134\230\031\162\000\041\306\215\154\031\327\365\037\140\230"



( run in 0.678 second using v1.01-cache-2.11-cpan-7add2cbd662 )