view release on metacpan or  search on metacpan

doc/recs-multiplex.pod  view on Meta::CPAN

=head1 NAME

recs-multiplex

=head1 recs-multiplex --help-all

 Help from: --help-basic:
 Usage: recs-multiplex <args> -- <other recs operation>
    Take records, grouped together by --keys, and run an operation for
    each group.
 
 Arguments:
    --key|-k <keys>              Comma separated list of key fields. May be a key
                                 spec or key group
    --dlkey|-K ...               Specify a domain language key. See "Domain
                                 Language Integration" section in --help-more.
    --size|--sz|-n <number>      Number of running clumps to keep.
    --adjacent|-1                Only group together adjacent records. Avoids
                                 spooling records into memeory
    --cube                       See "Cubing" section in --help-more.
    --clumper ...                Use this clumper to group records. May be
                                 specified multiple times. See --help-clumping.
    --dlclumper ...              Use this domain language clumper to group
                                 records. May be specified multiple times. See
                                 --help-clumping.
    --line-key|-L <key>          Use the value of this key as line input for the
                                 nested operation (rather than the entire
                                 record). Use with recs-from* scripts generally.
    --list-clumpers              Bail and output a list of clumpers
    --show-clumper <clumper>     Bail and output this clumper's detailed usage.
    --filename-key|fk <keyspec>  Add a key with the source filename (if no
                                 filename is applicable will put NONE)
 
   Help Options:
       --help-all             Output all help for this script
       --help                 This help screen
       --help-clumping        Help on clumping; mechanisms to group records
                              across a stream
       --help-domainlanguage  Help on the recs domain language, a [very
                              complicated] way of specifying valuations (which
                              act like keys) or aggregators
       --help-keygroups       Help on keygroups, a way of specifying
                              multiple keys
       --help-keys            Help on keygroups and keyspecs
       --help-keyspecs        Help on keyspecs, a way to index deeply and
                              with regexes
       --help-more            Larger help documentation
 
 Examples:
    Separate out a stream of text by PID into separate invocations of recs-frommultire.
       recs-fromre '^(.*PID=([0-9]*).*)$' -f line,pid | recs-multiplex -L line -k pid -- recs-frommultire ...
   Tag lines with counts by thread
       recs-multiplex -k thread -- recs-eval '{{nbr}} = ++$nbr'
 
 Help from: --help-clumping:
 CLUMPING:
    "Clumping" defines a way of taking a stream of input records and rearranging
    them into to groups for consideration. The most common "consideration" for
    such a group of records is the application of one or more aggregators by recs-
    collate and the most common clumpers are those specifiable by recs-collate's
    normal options. However, other recs scripts can use "clumpers" and much more
    complex clumping is possible. A list of clumpers can be found via the --list-
    clumpers option on recs-collate and documentation for individual clumpers can
    be inspected via --show-clumper.
 
 Examples:
    Group adjacent records for each host and output each such group's size.
       recs-collate -c keylru,host,1 -a ct
    Output the successive differences of the time field.
       recs-collate -c window,2 --dla 'time_delta=xform(recs, <<{{#1/time}} - {{#0/time}}>>)'
 
 Full list:
    cubekeyperfect: clump records by the value for a key, additionally cubing them
    keylru: clump records by the value for a key, limiting number of active clumps
    keyperfect: clump records by the value for a key
    window: clump records by a rolling window
 
 Help from: --help-domainlanguage:
 DOMAIN LANGUAGE
    The normal mechanism for specifying keys and aggregators allows one to
    concisely instantiate the objects that back them in the platform and is
    certainly the easiest way to use recs. The record stream domain language
    allows the creation of these objects in a programmatic way, with neither the
    syntactic issues of the normal way nor its guiding hand.
 
    The domain language is itself just Perl with a collection of library
    functions for creating platform objects included. Your favorite aggregators
    are all here with constructors matching their normal token. For convenience
    of e.g. last, aggregators are also included with a prefixed underscore.
 
    Below you can find documentation on all the "built in" functions. Most
    aggregators and deaggregators should be present with arguments comparable to
    their normal instantiation arugments, but with keyspec parameters replaced
    with valuations parameters.
 
 Special Syntax
    Where one sees a <snippet> argument below, a string scalar is expected,
    however quoting these can get fairly difficult and they can be confused with
    non-<snippet> scalars.
 
    Example:
      --dla "silly= uconcat(',', snip('{{x}} * 2'))"
 
    To remedy this, one may use <<CODE>> to inline a snippet which will be
    immediately understood by the typing mechanism as being code. Escaping inside
    this is as single quotes in Perl.
 
    Example With <<CODE>>
      --dla 'silly= uconcat(",", <<{{x}} * 2>>)'
 
    Furthermore one may mark variables to be propagated in by prefixing CODE like
    <<var1,var2,var3|CODE>>:
      --dla 'silly= $f=2; uconcat(",", <<f|{{x}} * $f>>)'

doc/recs-multiplex.pod  view on Meta::CPAN

    Fuzzy matching works like this in order, first key to match wins
      1. Exact match ( eq )
      2. Prefix match ( m/^/ )
      3. Match anywehre in the key (m//)
 
    So, in the above example '@b/#2', the 'b' portion would expand to 'biz' and 2
    would be the index into the array, so all records would have the value of 'c'
 
    Simiarly, @f/b would have values 1, 2, and 3
 
    You can escape / with a \. For example, if you have a record:
    {"foo/bar":2}
 
    You can address that key with foo\/bar
 
 Help from: --help-more:
 Usage: recs-multiplex <args> -- <other recs operation>
    Take records, grouped together by --keys, and run an operation for
    each group.
 
 Arguments:
    --key|-k <keys>              Comma separated list of key fields. May be a key
                                 spec or key group
    --dlkey|-K ...               Specify a domain language key. See "Domain
                                 Language Integration" section in --help-more.
    --size|--sz|-n <number>      Number of running clumps to keep.
    --adjacent|-1                Only group together adjacent records. Avoids
                                 spooling records into memeory
    --cube                       See "Cubing" section in --help-more.
    --clumper ...                Use this clumper to group records. May be
                                 specified multiple times. See --help-clumping.
    --dlclumper ...              Use this domain language clumper to group
                                 records. May be specified multiple times. See
                                 --help-clumping.
    --line-key|-L <key>          Use the value of this key as line input for the
                                 nested operation (rather than the entire
                                 record). Use with recs-from* scripts generally.
    --list-clumpers              Bail and output a list of clumpers
    --show-clumper <clumper>     Bail and output this clumper's detailed usage.
    --filename-key|fk <keyspec>  Add a key with the source filename (if no
                                 filename is applicable will put NONE)
 
   Help Options:
       --help-all             Output all help for this script
       --help                 This help screen
       --help-clumping        Help on clumping; mechanisms to group records
                              across a stream
       --help-domainlanguage  Help on the recs domain language, a [very
                              complicated] way of specifying valuations (which
                              act like keys) or aggregators
       --help-keygroups       Help on keygroups, a way of specifying
                              multiple keys
       --help-keys            Help on keygroups and keyspecs
       --help-keyspecs        Help on keyspecs, a way to index deeply and
                              with regexes
       --help-more            Larger help documentation
 
 Examples:
    Separate out a stream of text by PID into separate invocations of recs-frommultire.
       recs-fromre '^(.*PID=([0-9]*).*)$' -f line,pid | recs-multiplex -L line -k pid -- recs-frommultire ...
   Tag lines with counts by thread
       recs-multiplex -k thread -- recs-eval '{{nbr}} = ++$nbr'
 
 Cubing:
    Instead of added one entry for each input record, we add 2 ** (number of key
    fields), with every possible combination of fields replaced with the default
    of "ALL". This is not meant to be used with --adjacent or --size. If our key
    fields were x and y then we'd get output for {x = 1, y = 2}, {x = 1, y =
    ALL}, {x = ALL, y = 2} and {x = ALL, y = ALL}.
 
 Domain Lanuage Integration:
    The normal mechanism for specifying keys and aggregators allows one to
    concisely instantiate the objects that back them in the platform and is
    certainly the easiest way to use recs. The record stream domain language
    allows the creation of these objects in a programmatic way, with neither the
    syntactic issues of the normal way nor its guiding hand.
 
    The domain language is itself just Perl with a collection of library
    functions for creating platform objects included. Your favorite aggregators
    are all here with constructors matching their normal token. For convenience
    of e.g. last, aggregators are also included with a prefixed underscore.
 
    Below you can find documentation on all the "built in" functions. Most
    aggregators and deaggregators should be present with arguments comparable to
    their normal instantiation arugments, but with keyspec parameters replaced
    with valuations parameters.
 
    Keys may be specified using the recs domain language. --dlkey requires an
    option of the format '<name>=<domain language code>'. --dlkey requires the
    code evaluate as a valuation.
 
    See --help-domainlanguage for a more complete description of its workings and
    a list of available functions.
 
    See the examples in the recs-collate help for a more gentle introduction.
 

=head1 SEE ALSO

=over

=item * See L<App::RecordStream> for an overview of the scripts and the system

=item * Run C<recs examples> or see L<App::RecordStream::Manual::Examples> for a set of simple recs examples

=item * Run C<recs story> or see L<App::RecordStream::Manual::Story> for a humorous introduction to RecordStream

=item * Every command has a C<--help> mode available to print out usage and
examples for the particular command, just like the output above.

=back