view release on metacpan or  search on metacpan

lib/Algorithm/ToNumberMunger.pm  view on Meta::CPAN

or the C<method=> field): C<none> 0, C<password> 1, C<keyboard-interactive> 2,
C<hostbased> 3, C<publickey> 4, C<gssapi-with-mic> 5, C<gssapi-keyex> 6, with
bare C<gssapi> aliased to C<gssapi-with-mic>. Numbered roughly
B<weakest-to-strongest> so "weaker credential than expected" is a monotone
feature; the ordering is a judgement call, not a registry. Numeric inputs are
not passed through.

=head2 amavis_category_enum

    { munger => 'amavis_category_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for the amavisd-new
content category, ordered clean-to-worst: C<clean> 0, C<oversized> 1,
C<unchecked> 2, C<spammy> 3, C<spam> 4, C<badheader> 5, C<banned> 6,
C<infected> 7, C<mtablocked> 8. The hyphenated spellings (C<bad-header>,
C<mta-blocked>) and the legacy C<virus> (for C<infected>) are accepted as
aliases. The Passed/Blocked I<action> itself is a L</bool>; this is the
finer-grained reason. Numeric inputs are not passed through.

=head2 systemd_result_enum

    { munger => 'systemd_result_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for a systemd unit
C<result> (as in C<Failed with result 'timeout'>): C<success> 0, C<protocol>
1, C<timeout> 2, C<exit-code> 3, C<signal> 4, C<core-dump> 5, C<watchdog> 6,
C<start-limit-hit> 7, C<oom-kill> 8, C<resources> 9. The underscore spellings
(C<exit_code>, C<core_dump>, C<start_limit_hit>, C<oom_kill>) are accepted as
aliases. Numeric inputs are not passed through.

=head2 clamav_result_enum

    { munger => 'clamav_result_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for a ClamAV per-target
verdict as logged by C<clamd> / clamav-milter: C<OK> 0, C<FOUND> 1, C<ERROR>
2. Small but exactly the signal worth flagging (a C<FOUND> line). Numeric
inputs are not passed through.

=head2 kerberos_etype_enum

    { munger => 'kerberos_etype_enum', default => -1 }

Named-map enum for the Kerberos ticket encryption type -- Windows events
4768/4769 (logged as hex, C<0x17>) and any other AD/Kerberos source. The
values are the RFC 3961 etype numbers, which B<are> the wire encoding, so
(unlike most of the enums here) a decimal input passes through unchanged: the
map exists only to resolve the hex spellings (C<0x17> => 23, C<0x12> => 18,
...) and the RFC/MIT names (C<rc4-hmac>/C<rc4>/C<arcfour-hmac> => 23,
C<aes256-cts-hmac-sha1-96>/C<aes256> => 18, C<aes128> => 17, C<des3> => 16,
C<des-cbc-md5> => 3, C<des-cbc-crc> => 1) onto that same number. The classic
use is flagging C<rc4-hmac> (0x17) as a downgrade/roasting signal against an
C<aes256> baseline. Lookup is case-insensitive; an unlisted etype croaks
unless a numeric C<default> is given.

=head2 windows_integrity_level_enum

    { munger => 'windows_integrity_level_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for the Windows / Sysmon
process integrity level, as an B<ordinal>: C<untrusted> 0, C<low> 1, C<medium>
2, C<high> 3, C<system> 4 (with C<mediumplus> folded into C<medium>). The
C<S-1-16-*> mandatory-label SIDs Windows sometimes logs in place of the word
(C<S-1-16-12288> => 3, ...) are accepted as aliases. Ordinal so "higher
privilege than expected" is a monotone feature. Numeric inputs are not passed
through.

=head2 windows_logon_status_enum

    { munger => 'windows_logon_status_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for the NTSTATUS
sub-status on a failed Windows logon (events 4625/4776), collapsed to a
compact B<reason category> rather than the raw 32-bit code: C<0xC0000064>
(no such user) 0, C<0xC000006A> (bad password) 1, C<0xC000006D> (generic bad
user/pass) 2, C<0xC000006F> (outside hours) 3, C<0xC0000070> (workstation
restriction) 4, C<0xC0000071> (password expired) 5, C<0xC0000072> (disabled)
6, C<0xC0000193> (account expired) 7, C<0xC0000133> (clock skew) 8,
C<0xC0000224> (must change password) 9, C<0xC0000234> (locked out) 10,
C<0xC000015B> (logon type not granted) 11. Keys are the hex codes exactly as
logged (matched case-insensitively); only the common logon subset is baked in,
so an unlisted code croaks unless a numeric C<default> is given. Numeric
inputs are not passed through.

=head2 windows_impersonation_level_enum

    { munger => 'windows_impersonation_level_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for the event 4624
impersonation level, ordered by reach: C<anonymous> 0, C<identification> 1,
C<impersonation> 2, C<delegation> 3. The C<%%1832> / C<%%1833> message tokens
Windows often emits in place of the words (Identification / Impersonation) are
accepted as aliases. Numeric inputs are not passed through.

=head2 aad_signin_error_enum

    { munger => 'aad_signin_error_enum', default => -1 }

Named-map enum for the Azure AD / Entra sign-in C<ResultType> error code,
collapsed to a compact B<reason category> rather than the raw code: C<0>
(success) 0, invalid-password (C<50126>, C<50056>) 1, no-such-user (C<50034>)
2, disabled (C<50057>) 3, locked / smart-lockout (C<50053>) 4, password-expired
(C<50055>, C<50144>) 5, MFA-required (C<50074>, C<50076>, C<50079>) 6,
MFA-failed (C<500121>, C<50158>) 7, blocked-by-conditional-access (C<53003>,
C<53000>, C<53001>, C<530032>) 8, session-expired (C<50173>) 9. Although
C<ResultType> is already numeric, the code space is huge and sparse and its
magnitude carries no signal -- this maps the common codes onto a handful of
meaningful buckets (and, unlike L</hash>, keeps related codes together). Keys
are the codes as logged; only the common subset is baked in, so an unlisted
code croaks unless a numeric C<default> is given. Because the output is a
category of our own numbering, numeric inputs are B<not> passed through.

=head2 risk_level_enum

    { munger => 'risk_level_enum', default => -1 }

Named-map enum (lookup rules as L</dns_rcode_enum>) for the Entra Identity
Protection C<riskLevel> as an B<ordinal>: C<none> 0, C<low> 1, C<medium> 2,
C<high> 3. C<hidden> and C<unknownFutureValue> are left to the C<default>
(they are not points on the scale). Numeric inputs are not passed through.

lib/Algorithm/ToNumberMunger.pm  view on Meta::CPAN

sub _build_chain {
	my ( $spec, $where ) = @_;

	my $ops = _chain_steps( $spec, $where );
	my ( $then, $name ) = _chain_terminal_spec( $spec, $where );
	my $builder = $BUILDERS{$name}
		or croak "chain munger$where: unknown terminal munger '$name' (known: "
		. join( ', ', sort keys %BUILDERS ) . ')';
	my $term = $builder->( $then, "$where (chain terminal)" );

	return sub {
		my $s = defined $_[0] ? "$_[0]" : '';
		$s = $_->($s) for @$ops;
		return $term->($s);
	};
} ## end sub _build_chain

# Multi-output chain: the same pre-transforms, but the terminal is one of the
# multi-output ('into') mungers. Returns ($list_returning_code, $arity) like
# every multi builder; the arity is the terminal's.
sub _build_chain_multi {
	my ( $spec, $where ) = @_;

	my $ops = _chain_steps( $spec, $where );
	my ( $then, $name ) = _chain_terminal_spec( $spec, $where );
	my $builder = $MULTI_BUILDERS{$name}
		or croak "chain munger$where: terminal munger '$name' does not support "
		. "multiple outputs ('into'); only these do: "
		. join( ', ', sort keys %MULTI_BUILDERS );
	my ( $term, $arity ) = $builder->( $then, "$where (chain terminal)" );

	my $code = sub {
		my $s = defined $_[0] ? "$_[0]" : '';
		$s = $_->($s) for @$ops;
		return $term->($s);
	};
	return ( $code, $arity );
} ## end sub _build_chain_multi

=head2 eps

    { munger => 'eps', prefix => 'http-req:', from => 'src_ip' }
    { munger => 'eps', prefix => 'dns-nxd:',  from => 'src_ip',
      read => 'rate', mark => 0 }
    # multi-output: one daemon round trip fills several columns
    { munger => 'eps', prefix => 'http-req:', from => 'src_ip',
      parts => [ 'rate', 'count' ], into => [ 'req_rate', 'req_count' ] }

Per-entity sliding-window event rates via the C<iqbi-damiq> daemon shipped with
L<Algorithm::EventsPerSecond> (see
L<Algorithm::EventsPerSecond::Sukkal>). The input value becomes a meter B<key>
(after C<prefix> is prepended); by default the munger B<marks> one event against
that key and returns the key's current events-per-second, using the daemon's
C<MARKRATE> command -- mark and query in a single command with a single reply.
This is the munger behind rate columns like a per-source request rate: every
event marks its source's meter and stores the rate the meter now reads.

Unlike every other munger this one consults external state -- but the state
lives in the daemon, not here, so the munger itself remains a stateless client
and rows stay reproducible I<given> the daemon. Because the daemon is shared,
multiple writer processes marking the same keys see one B<global> rate, which an
in-process meter could never give.

Spec keys:

=over 4

=item * C<socket> - unix socket path of the daemon. Defaults to
C<$Algorithm::ToNumberMunger::EPS_SOCKET>
(C</var/run/iqbi-damiq.sock>).

=item * C<prefix> - string prepended to the input to form the key, namespacing
this column's meters (two columns keyed on the same field need different
prefixes or they share meters). No whitespace/control characters. Default C<''>.

=item * C<mark> - whether to mark the key (default C<1>). Marking rides
C<MARKRATE>: with C<< read => 'rate' >> that one command is the whole exchange;
with C<count>/C<total> the read is pipelined after it (the C<MARKRATE> rate
reply is discarded), so a marking failure still comes back as an ordinary
first reply. With C<< mark => 0 >> the munger only reads, for columns whose
marking is done elsewhere -- e.g. an NXDOMAIN rate is I<marked> by the pipeline
only on NXDOMAIN responses but I<read> on every row.

=item * C<read> - what to read: C<rate> (events/sec over the daemon's window,
default), C<count> (events inside the window), or C<total> (lifetime).

=item * C<parts> + C<into> - multi-output form (see L</compile>): read several
of C<rate>/C<count>/C<total> for the one key in a single round trip, filling one
column each. When marking, the C<MARKRATE> reply itself serves the first C<rate>
part, so C<< parts => ['rate', 'count'] >> costs exactly two commands.

=item * C<on_error> - C<'die'> (default) croaks the write when the daemon is
unreachable or replies C<ERR>; a number is returned instead as a quiet fallback.
Note C<0> is indistinguishable from a genuinely idle key, so quiet fallback
biases the column -- loud is the default on purpose.

=item * C<timeout> - per-operation socket timeout in seconds (default 5,
best-effort via C<SO_RCVTIMEO>/C<SO_SNDTIMEO>), so a wedged daemon cannot hang a
writer forever.

=back

Semantics worth knowing: a marked read B<includes the event just marked>; keys
have whitespace/control bytes replaced with C<_> to satisfy the daemon's key
rules; connections are made lazily on first use and kept open (reconnecting
transparently after a fork or an error), so compiling a plan -- including the
eager validation in C<write_info> -- needs no running daemon. Each eps column
costs one unix-socket round trip per row; the multi-output form exists so
rate+count of the same key costs one round trip, not two.

=cut

# Default socket path of the iqbi-damiq daemon.
our $EPS_SOCKET = '/var/run/iqbi-damiq.sock';

# Persistent daemon connections, keyed by socket path, shared by every eps
# munger in the process. Entries record the pid that opened them so a forked
# writer transparently reopens instead of sharing a socket with its parent.
# Connections are made lazily on first use -- never at munger build time, so a
# plan can compile (eager validation) with no daemon running.
my %EPS_CONN;

sub _eps_conn {
	my ( $path, $timeout ) = @_;
	my $c = $EPS_CONN{$path};
	return $c->{fh} if $c && $c->{pid} == $$;

	require Socket;
	require IO::Socket::UNIX;
	my $fh = IO::Socket::UNIX->new(
		Type => Socket::SOCK_STREAM(),
		Peer => $path,
	) or die "cannot connect to iqbi-damiq at $path: $!\n";

	# Best-effort read/write timeouts so a wedged daemon cannot hang a writer.
	eval {
		my $tv = pack( 'l!l!', $timeout, 0 );
		setsockopt( $fh, Socket::SOL_SOCKET(), Socket::SO_RCVTIMEO(), $tv );
		setsockopt( $fh, Socket::SOL_SOCKET(), Socket::SO_SNDTIMEO(), $tv );
	};

	$EPS_CONN{$path} = { fh => $fh, pid => $$ };
	return $fh;
} ## end sub _eps_conn

# One pipelined transaction: send $cmd (possibly several lines) and read
# $nreplies "OK n" lines, one per command sent. The munger only ever sends
# commands that reply exactly once -- MARKRATE (which marks AND returns the
# rate in one go), RATE, COUNT, TOTAL; never a bare MARK, whose reply-only-on-
# error behavior would let a failure desynchronize the reply stream. Dies on
# ERR, EOF, or timeout; the caller still drops the cached connection on error
# as belt and braces.
sub _eps_txn {
	my ( $path, $timeout, $cmd, $nreplies ) = @_;
	my $fh = _eps_conn( $path, $timeout );
	print {$fh} $cmd or die "write to iqbi-damiq failed: $!\n";
	my @out;
	for ( 1 .. $nreplies ) {
		my $reply = <$fh>;
		die "iqbi-damiq closed the connection (or timed out)\n"
			unless defined $reply;
		$reply =~ /\AOK (\S+)/
			or die "iqbi-damiq replied: $reply";
		push @out, $1 + 0;
	}
	return @out;
} ## end sub _eps_txn

# Validate the spec keys shared by the scalar and multi-output eps builders.
sub _eps_spec {
	my ( $spec, $where ) = @_;

	my $socket = defined $spec->{socket} ? $spec->{socket} : $EPS_SOCKET;
	croak "eps munger$where: 'socket' must be a non-empty path"
		unless length $socket;

	my $prefix = defined $spec->{prefix} ? $spec->{prefix} : '';