Algorithm-Classifier-IsolationForest
view release on metacpan or search on metacpan
nan :: range over present values and route missing rows to the
right child, consistently at fit and score time
- new `impute_with => 'mean'|'median'` option for impute mode ...
missing strategy + impute fill vector are persisted in saved models;
models from older releases load as `zero` (the prior undef -> 0
scoring behaviour)
- the C build is now tunable via environment variables read at first
module load: IF_ARCH=<value> adds -march=<value>, IF_NATIVE=1 is
shorthand for IF_ARCH=native, IF_OPT overrides the default -O3,
and IF_NO_C=1 skips building the C backend entirely; values are
validated (bad ones warn and fall back to the defaults) and the
flags actually used are exposed via $OPT_LEVEL
- Benchmarking: bench-sklearn-scoring.pl now compares pure Perl, C,
and C+OpenMP fit/score paths against sklearn side by side
0.2.1 2026-06-30/14:30
- derp... actually update MANIFEST so a bunch of files from last release
are actually included
0.2.0 2026-06-30/14:15
- C acceleration via Inline::C for core fit and predict ops
benchmarking/bench-online-score-accel.pl view on Meta::CPAN
#!/usr/bin/perl
# benchmarking/bench-online-score-accel.pl
#
# Benchmarks Online Isolation Forest batch scoring under each acceleration
# backend:
# pure_perl -- use_c => 0 (pure Perl tree walk)
# c_serial -- use_c => 1, use_openmp => 0 (C tree walk, single thread)
# c_openmp -- use_c => 1, use_openmp => 1 (C tree walk, OpenMP parallel)
#
# The online class scores through the parent's C backend by lazily packing
# its mutable trees into the parent's node layout; learning invalidates the
# packed snapshot and the next scoring call repacks once. Learning itself
# (and the per-row walks inside score_learn) runs in C directly against the
# live trees. Sections 1 and 2 measure steady-state batch scoring (snapshot
# reused across calls); section 3 interleaves a learned point before every
# scoring call, so each call pays the repack -- the worst case for the C
# path; sections 4 and 5 measure the prequential score_learn /
# score_learn_tagged stream loop, where the mutable-tree C walks are what
# matters.
#
# Reference numbers (2026-07-08, 8-core dev box, 100 trees, window 2048,
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
}
# -O3 is the usual default: it's safe to enable unconditionally and
# matters here -- the extended-mode oblique dot product is wrapped in
# `#pragma omp simd`, but without aggressive optimization the compiler
# may still emit scalar code. Use OPTIMIZE (not CCFLAGS) -- CCFLAGS is
# prepended to the cc line and would be shadowed by Perl's own `-O2 -g`
# that ExtUtils::MakeMaker appends afterward (last `-O` wins in gcc).
# IF_OPT overrides the level itself (e.g. IF_OPT=-O2 to work around a
# miscompile, or to shorten build time while developing); it's
# validated against a fixed set of GCC/Clang -O flags rather than
# interpolated as-is, since this string eventually reaches a shell
# command line via ExtUtils::MakeMaker.
my $opt = $def_opt;
if ( defined $ENV{IF_OPT} ) {
if ( $ENV{IF_OPT} =~ /\A-O[0123sgz]\z/ ) {
$opt = $ENV{IF_OPT};
} else {
warn "Algorithm::Classifier::IsolationForest: ignoring invalid "
. "IF_OPT value '$ENV{IF_OPT}' (expected one of -O0 -O1 -O2 "
. "-O3 -Os -Og -Oz); using $opt\n";
}
}
# -march=<value> lets the compiler target specific instruction-set
# extensions (AVX2 gather + FMA, etc.) for the oblique dot product
# and the fit-time min/max scan's `#pragma omp simd` loops.
#
# IF_ARCH=<value> sets it explicitly (e.g. "x86-64-v3", "skylake",
# "znver3") -- validated against a conservative identifier charset
# since, like IF_OPT, it flows into a compiler command line.
# IF_NATIVE=1 remains as shorthand for IF_ARCH=native and is used
# when IF_ARCH isn't set. Prefer a specific IF_ARCH value over
# IF_NATIVE on a machine you don't control exclusively: blanket
# -march=native pulls in whatever the build host has, including
# AVX-512 on some Intel CPUs, which is known to trigger clock
# throttling under sustained heavy use and can make throughput
# *worse* than a conservative target like x86-64-v3 (AVX2, no
# AVX-512). Either way, the cached artefact under _Inline/ is then
# pinned to that instruction set, so leave both unset if the
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
the default C<-O3>, e.g. to shorten build time while iterating, or work
around a miscompile on an unusual toolchain. Invalid values are ignored
with a warning rather than passed through, since this string reaches a
compiler command line.
=item * C<IF_ARCH=E<lt>valueE<gt>> -- adds C<-march=E<lt>valueE<gt>> so the
compiler can target specific instruction-set extensions (AVX2 gather +
FMA, etc.) for the extended-mode oblique dot product and the fit-time
min/max scan's C<#pragma omp simd> loops. Accepts values like
C<x86-64-v3>, C<skylake>, or C<znver3> -- whatever your compiler's
C<-march=> accepts. Also validated (a restricted character set, not
passed through as-is) for the same reason as C<IF_OPT>. The special
value C<none> (or an empty string) opts out of any arch recorded at
configure time, yielding a plain build. Whenever a C<-march> is in
effect the build also adds C<-ffp-contract=off>: with FMA available
the compiler would otherwise contract C<a*b+c> into fused
multiply-adds whose different rounding breaks the guarantee that
C<use_c =E<gt> 1> and C<use_c =E<gt> 0> build bit-identical trees (the
C<-march> speedup comes from vectorization, not contraction, so this
costs essentially nothing).
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# travels with the model file.
schema_version => $args{schema_version},
schema_description => $args{schema_description},
feature_descriptions => $args{feature_descriptions},
};
for my $doc (qw(schema_version schema_description)) {
croak "$doc must be a plain string"
if defined $self->{$doc} && ref $self->{$doc};
}
_validate_feature_descriptions( $self->{feature_names}, $self->{feature_descriptions} )
if defined $self->{feature_descriptions};
# Optional Algorithm::ToNumberMunger integration: a declarative spec
# hash compiled into a plan that turns raw tagged values into numbers.
# Compiled eagerly so every spec error surfaces here rather than at
# first scoring; the module itself is only required when a spec is
# actually given, keeping it an optional dependency.
if ( defined $args{mungers} ) {
croak "mungers must be a hashref of 'tag => munger spec'"
unless ref $args{mungers} eq 'HASH';
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# knobs are deliberately absent from the params lists.
my %PROTO_PARAM_KEYS = (
batch => { map { $_ => 1 } qw(n_trees sample_size max_depth mode extension_level contamination voting seed) },
online => { map { $_ => 1 } qw(n_trees window_size max_leaf_samples growth subsample contamination seed) },
);
my %PROTO_SCHEMA_KEYS = (
batch => { map { $_ => 1 } qw(feature_names feature_descriptions mungers missing impute_with) },
online => { map { $_ => 1 } qw(feature_names feature_descriptions mungers missing) },
);
=head2 validate_prototype($proto)
Structurally validates a prototype -- a hashref or a JSON string -- and
returns the decoded hashref; croaks describing the first problem found.
Validation is structural only (no munger compilation), so it does not
require Algorithm::ToNumberMunger even for a munger-bearing prototype.
my $proto = Algorithm::Classifier::IsolationForest->validate_prototype($json);
=cut
sub validate_prototype {
my ( $class, $proto ) = @_;
if ( !ref $proto ) {
my $decoded = eval { JSON::PP->new->decode($proto) };
croak "prototype did not parse as JSON: $@" if $@;
$proto = $decoded;
}
croak "not an IsolationForest prototype (expected a JSON object)"
unless ref $proto eq 'HASH';
croak "not an IsolationForest prototype (format is not " . "'Algorithm::Classifier::IsolationForest::Prototype')"
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
. join( ', ', sort keys %{ $PROTO_SCHEMA_KEYS{$which} } ) . ')'
unless $PROTO_SCHEMA_KEYS{$which}{$k};
}
my $tags = $schema->{feature_names};
croak "prototype schema needs a non-empty feature_names array"
unless ref $tags eq 'ARRAY' && @$tags;
for my $t (@$tags) {
croak "prototype feature_names entries must be non-empty strings"
unless defined $t && !ref $t && length $t;
}
_validate_feature_descriptions( $tags, $schema->{feature_descriptions} )
if defined $schema->{feature_descriptions};
croak "prototype schema mungers must be an object of 'tag => munger spec'"
if defined $schema->{mungers} && ref $schema->{mungers} ne 'HASH';
for my $str (qw(missing impute_with)) {
croak "prototype schema $str must be a plain string"
if defined $schema->{$str} && ref $schema->{$str};
}
my $params = $proto->{params};
croak "prototype params must be an object of tuning knobs"
if defined $params && ref $params ne 'HASH';
for my $k ( sort keys %{ $params || {} } ) {
croak "prototype params has unknown key '$k' for a $which prototype (allowed: "
. join( ', ', sort keys %{ $PROTO_PARAM_KEYS{$which} } )
. '; machine-local knobs like use_c are deliberately not allowed)'
unless $PROTO_PARAM_KEYS{$which}{$k};
}
return $proto;
} ## end sub validate_prototype
=head2 new_from_prototype($proto, %overrides)
Creates a fresh, unfitted model from a prototype (a hashref or a JSON
string) and returns it -- an instance of whichever class the prototype's
C<class> field names, so like C<load()> this is a single entry point for
both model types. Croaks on any validation failure; a munger-bearing
prototype compiles its plan here, so a bogus munger spec dies at
creation (and needs Algorithm::ToNumberMunger installed).
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
my $oif = Algorithm::Classifier::IsolationForest->new_from_prototype(
$proto_json,
seed => 42,
);
=cut
sub new_from_prototype {
my ( $class, $proto, %overrides ) = @_;
$proto = $class->validate_prototype($proto);
my $which = $proto->{class};
my $schema = $proto->{schema};
for my $k ( sort keys %overrides ) {
croak "new_from_prototype: '$k' is part of the prototype's schema and may not "
. "be overridden; edit the prototype instead"
if $k
=~ /\A(?:feature_names|feature_descriptions|mungers|missing|impute_with|schema_version|schema_description)\z/;
croak "new_from_prototype: unknown override '$k' for a $which prototype (allowed: "
. join( ', ', sort keys %{ $PROTO_PARAM_KEYS{$which} } ) . ')'
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# class's delegating methods can hand them their own $self: the plan and
# spec live in the same hash slots on either class.
# ---------------------------------------------------------------------------
# Validate a feature_descriptions hash against the feature names: every
# described feature must exist (a description for one that does not is
# either a typo or a stale leftover from a schema change) and every
# description must be a plain string. Partial coverage is fine. A plain
# function, like the munger helpers, so both classes and the prototype
# validator can call it.
sub _validate_feature_descriptions {
my ( $tags, $fd ) = @_;
croak "feature_descriptions must be a hashref of 'feature name => description'"
unless ref $fd eq 'HASH';
croak "feature_descriptions requires feature_names to validate against"
unless ref $tags eq 'ARRAY' && @$tags;
my %known = map { $_ => 1 } @$tags;
for my $k ( sort keys %$fd ) {
croak "feature_descriptions describes '$k', which is not in feature_names"
unless $known{$k};
croak "feature_descriptions entry for '$k' must be a plain string"
if ref $fd->{$k};
}
return 1;
} ## end sub _validate_feature_descriptions
# Compile a munger spec against the model's feature names. Requires
# Algorithm::ToNumberMunger on demand -- it is an optional dependency --
# and lets its compile() croak on any spec problem.
sub _compile_mungers {
my ( $tags, $mungers ) = @_;
croak "this model has mungers but no feature_names to compile them against"
unless ref $tags eq 'ARRAY' && @$tags;
local $@;
eval { require Algorithm::ToNumberMunger; 1 }
lib/Algorithm/Classifier/IsolationForest/App/Command.pm view on Meta::CPAN
package Algorithm::Classifier::IsolationForest::App::Command;
use strict;
use warnings;
use App::Cmd::Setup -command;
sub global_opt_spec {
my ( $class, $app ) = @_;
return ( $class->options($app), );
}
sub validate_args {
my ( $self, $opt, $args ) = @_;
if ( $opt->{help} ) {
my ($command) = $self->command_names;
$self->app->execute_command( $self->app->prepare_command( "help", $command ) );
exit;
}
$self->validate( $opt, $args );
}
return 1;
lib/Algorithm/Classifier/IsolationForest/App/Command/accel.pm view on Meta::CPAN
* IF_NO_OPENMP=1 -- serial C backend: no libgomp linkage at all
* IF_RUNTIME_BUILD=1 -- ignore the prebuilt object, compile at first load
* IF_NO_C=1 -- skip the C backend entirely
See "NATIVE ACCELERATION" in perldoc Algorithm::Classifier::IsolationForest
for details and tradeoffs (in particular, why IF_NATIVE is not always a
safe default choice).
';
} ## end sub description
sub validate { 1 }
sub execute {
my ( $self, $opt, $args ) = @_;
# Tiny deterministic dataset. Fitting + scoring confirms the chosen
# backend is callable end-to-end, not merely that it compiled. We
# exercise both axis mode (covers score_all_xs's axis branch) and
# extended mode (covers the oblique branch -- where the
# `#pragma omp simd` reduction lives, so this is the only path
# SIMD actually matters for).
lib/Algorithm/Classifier/IsolationForest/App/Command/bench.pm view on Meta::CPAN
Use this to answer:
* is my Inline::C / OpenMP / SIMD build actually faster than the
pure-Perl fallback?
* how much does pack_data help on my data shape?
* what is the per-call throughput I can expect at production-typical
query-set sizes?
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined $opt->{'i'} ) {
$self->usage_error('-i has not been specified');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/bench.pm view on Meta::CPAN
if ( $opt->{'secs'} <= 0 ) {
$self->usage_error('--secs must be > 0');
}
if ( $opt->{'t'} <= 0 || $opt->{'t'} >= 1 ) {
$self->usage_error('-t must satisfy 0 < t < 1');
}
return 1;
} ## end sub validate
# Standard bench helper: warm up briefly, then time exactly $secs of
# back-to-back calls. Returns ops/second.
sub _bench {
my ( $code, $secs ) = @_;
my $t0 = time();
$code->() while time() - $t0 < 0.3;
$t0 = time();
my $n = 0;
$code->(), $n++ while time() - $t0 < $secs;
lib/Algorithm/Classifier/IsolationForest/App/Command/csv2plot.pm view on Meta::CPAN
3range: Use columns 1 and 2 for x/y, and the second-to-last column for the
score gradient. Suitable for predict -d output (any number of features).
3binary: Use columns 1 and 2 for x/y, and the last column for normal/abnormal.
Suitable for predict -d output or gblob output.
3range and 3binary require data outputted from predict with the -d flag.
For N-dimensional data, columns 1 and 2 are always used for the x/y axes.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined( $opt->{'i'} ) ) {
$self->usage_error('-i has not been specified for a file to process');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/csv2plot.pm view on Meta::CPAN
if ( ( $opt->{'p'} ne 'auto' )
&& ( $opt->{'p'} ne '2heat' )
&& ( $opt->{'p'} ne '3range' )
&& ( $opt->{'p'} ne '3binary' ) )
{
$self->usage_error( '-p, "' . $opt->{'p'} . '", is not set to auto, 2heat, 3range, or 3binary' );
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
my @raw_csv = read_file( $opt->{'i'} );
my @first_fields = split( /,/, $raw_csv[0] );
my $ncols = scalar @first_fields;
if ( $opt->{'p'} eq 'auto' && $ncols >= 4 ) {
$opt->{'p'} = '3range';
lib/Algorithm/Classifier/IsolationForest/App/Command/fit.pm view on Meta::CPAN
--voting -> voting
With --prototype the schema (feature names, descriptions, mungers,
missing policy) and schema_version/schema_description come from the
prototype file, its params supply knob defaults, and the switches above
override those params. See PROTOTYPES in the module POD for the format.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined( $opt->{'i'} ) ) {
$self->usage_error('-i has not been specified');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/fit.pm view on Meta::CPAN
} elsif ( !-r $opt->{'prototype'} ) {
$self->usage_error( '--prototype, "' . $opt->{'prototype'} . '", is not readable' );
}
if ( defined( $opt->{'t'} ) || defined( $opt->{'mungers'} ) ) {
$self->usage_error(
'--prototype may not be combined with -t or --mungers; the schema comes only from the prototype');
}
} ## end if ( defined( $opt->{'prototype'} ) )
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
my $mode = 'axis';
if ( $opt->{'extended'} ) {
$mode = 'extended';
}
# Munger spec, decoded up front so a bad file dies before the CSV is
lib/Algorithm/Classifier/IsolationForest/App/Command/fit.pm view on Meta::CPAN
if ( defined( $opt->{'mungers'} ) ) {
require JSON::PP;
$mungers = eval { JSON::PP->new->decode( scalar read_file( $opt->{'mungers'} ) ) };
die( '--mungers, "' . $opt->{'mungers'} . '", did not parse as JSON: ' . $@ ) if $@;
die( '--mungers, "' . $opt->{'mungers'} . '", must be a JSON object of tag => spec' )
unless ref $mungers eq 'HASH';
}
# A prototype supplies the schema and knob defaults, so the model is
# created before the CSV is read (a munger-bearing prototype changes
# how the CSV is validated). Explicit tuning switches override the
# prototype's params; the schema may not be overridden at all.
my $iforest;
if ( defined( $opt->{'prototype'} ) ) {
my $proto = eval {
Algorithm::Classifier::IsolationForest->validate_prototype( scalar read_file( $opt->{'prototype'} ) );
};
die( '--prototype, "' . $opt->{'prototype'} . '", is not a valid prototype: ' . $@ ) if $@;
die( '--prototype, "' . $opt->{'prototype'} . '", is for an online model; use `iforest stream`' . "\n" )
unless $proto->{class} eq 'batch';
my %overrides;
$overrides{'n_trees'} = $opt->{'n'} if defined $opt->{'n'};
$overrides{'seed'} = $opt->{'s'} if defined $opt->{'s'};
$overrides{'sample_size'} = $opt->{'m'} if defined $opt->{'m'};
$overrides{'max_depth'} = $opt->{'d'} if defined $opt->{'d'};
lib/Algorithm/Classifier/IsolationForest/App/Command/gblob.pm view on Meta::CPAN
$truth is a 0/1 with 1 meaning it is a abnormal value.
Normal points are drawn from N(0,1) in each dimension. Anomalous points are
placed on a hyperspherical shell at radius 5-8 from the origin.
Use -D to control the number of dimensions (default: 2).
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( defined( $opt->{'s'} ) && $opt->{'s'} <= 0 ) {
$self->usage_error( '-s, "' . $opt->{'s'} . '", is less than or equal to 0, should be a positive int' );
}
if ( !$opt->{'p'} && -e $opt->{'o'} && !$opt->{'w'} ) {
$self->usage_error(
'-o "' . $opt->{'o'} . '", already exists. Specify -w to overwrite it or use a different value.' );
}
if ( $opt->{'n'} < 1 ) {
$self->usage_error( '-n, "' . $opt->{'n'} . '", must be be 1 or greater' );
}
if ( $opt->{'d'} < 1 ) {
$self->usage_error( '-D, "' . $opt->{'d'} . '", must be 1 or greater' );
}
return 1;
} ## end sub validate
sub gaussian {
my ( $mu, $sigma ) = @_;
my $u1 = rand() || 1e-12;
my $u2 = rand();
return $mu + $sigma * sqrt( -2 * log($u1) ) * cos( 2 * PI * $u2 );
}
sub execute {
my ( $self, $opt, $args ) = @_;
lib/Algorithm/Classifier/IsolationForest/App/Command/info.pm view on Meta::CPAN
sub description {
'Loads a saved Algorithm::Classifier::IsolationForest model and prints the
constructor params, fit-time metadata, and a handful of derived tree
statistics (count, average/max depth, total nodes).
Use --json for a machine-readable dump suitable for piping into jq.
'
}
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !-f $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not readable' );
}
return 1;
} ## end sub validate
# Tree-shape stats are derived once at load time. Each tree is a
# nested arrayref structure -- leaf [0, size] or interior [1, ...] /
# [2, ...] with children at fixed slots.
sub _walk_tree {
my ( $node, $depth, $acc ) = @_;
$acc->{nodes}++;
if ( $node->[0] == 0 ) { # leaf
$acc->{leaves}++;
$acc->{max_depth} = $depth if $depth > $acc->{max_depth};
lib/Algorithm/Classifier/IsolationForest/App/Command/pack.pm view on Meta::CPAN
my $magic;
my $ok = read( $fh, $magic, 8 ) == 8;
close $fh;
return $ok && $magic eq MAGIC;
}
sub opt_spec {
return (
[
'm=s',
'Model JSON to validate n_features against.',
{ 'default' => 'iforest_model.json', 'completion' => 'files' }
],
[ 'i=s', 'Input CSV to pack.', { 'completion' => 'files' } ],
[ 'o=s', 'Output .iforest-packed file path.', { 'completion' => 'files' } ],
[ 'w', 'Overwrite -o if it already exists.' ],
);
} ## end sub opt_spec
sub abstract { 'Pre-pack a CSV dataset into a binary file the scoring commands can read directly' }
sub description {
'Reads a CSV, validates that every row has the same numeric
column count as the model expects, runs the data through pack_data, and
writes a self-contained binary (.iforest-packed) the other iforest
commands can consume directly.
This skips the CSV parse + pack_input_xs cost on subsequent scoring
runs. It is most useful when the same data set is scored repeatedly
with different thresholds, e.g. during interactive tuning:
iforest pack -m model.json -i data.csv -o data.packed
iforest predict -m model.json -i data.packed -t 0.55 -o pred-55.csv
lib/Algorithm/Classifier/IsolationForest/App/Command/pack.pm view on Meta::CPAN
iforest predict -m model.json -i data.packed -t 0.75 -o pred-75.csv
The file format begins with the magic bytes "IFPKD\0\0\0". predict
auto-detects it on its -i input.
Requires the Inline::C backend; pure-Perl installs cannot produce or
consume the packed format.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined $opt->{'i'} ) {
$self->usage_error('-i has not been specified');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/pack.pm view on Meta::CPAN
$self->usage_error( '-o, "' . $opt->{'o'} . '", already exists and -w was not specified' );
}
if ( !-f $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not readable' );
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
die "iforest pack requires the Inline::C backend\n"
unless $Algorithm::Classifier::IsolationForest::HAS_C;
my $model = Algorithm::Classifier::IsolationForest->load( $opt->{'m'} );
my $nf = $model->{n_features};
lib/Algorithm/Classifier/IsolationForest/App/Command/predict.pm view on Meta::CPAN
$score,$predict
If -d is specified all input feature columns are prepended. When the
input is a .iforest-packed file the columns come from unpacking the
stored doubles.
$feat1,...,$featN,$score,$predict
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined( $opt->{'i'} ) ) {
$self->usage_error('-i has not been specified for a file to process');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/predict.pm view on Meta::CPAN
$self->usage_error( '-o, "' . $opt->{'o'} . '", already exists and -w is not specified' );
}
if ( defined( $opt->{'t'} ) && $opt->{'t'} <= 0 ) {
$self->usage_error( '-t, "' . $opt->{'t'} . '", needs to be greater than 0 and less than 1' );
} elsif ( defined( $opt->{'t'} ) && $opt->{'t'} >= 1 ) {
$self->usage_error( '-t, "' . $opt->{'t'} . '", needs to be greater than 0 and less than 1' );
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
my $iforest = Algorithm::Classifier::IsolationForest->load( $opt->{'m'} );
# A model carrying Algorithm::ToNumberMunger specs takes raw values in
# its munged CSV columns: skip the per-field numeric check at read
# time and munge the rows before scoring (re-checking numerics after).
# Packed input is never munged -- it is already doubles.
lib/Algorithm/Classifier/IsolationForest/App/Command/proto.pm view on Meta::CPAN
],
[
'o=s',
'Output the prototype to this file instead of printing (--from-model only).',
{ 'completion' => 'files' }
],
[ 'w', 'If the file specified via -o exists, over write it.' ],
);
} ## end sub opt_spec
sub abstract { 'Extract a prototype from a saved model, or validate a prototype file' }
sub description {
'Works with model prototypes: small JSON documents holding the variable
schema (feature names, per-feature descriptions, munger specs, missing
policy), a user-owned schema_version and schema_description, and
optionally the tuning knobs. `iforest fit --prototype` and
`iforest stream --prototype` create models from one; see PROTOTYPES in
the Algorithm::Classifier::IsolationForest POD for the file format.
--from-model extracts a prototype from a saved model, closing the loop:
pull the schema and knobs out of a good model, edit the metadata, and
create fresh models from it. A model with no recorded schema_version /
schema_description gets placeholder values to edit in.
--check validates a prototype file and prints a summary of what it
describes, exiting non-zero when the file is not a valid prototype.
Exactly one of --from-model or --check must be given.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
my $from = defined $opt->{'from_model'} ? 1 : 0;
my $check = defined $opt->{'check'} ? 1 : 0;
if ( $from + $check != 1 ) {
$self->usage_error('exactly one of --from-model or --check must be specified');
}
my ( $switch, $file ) = $from ? ( '--from-model', $opt->{'from_model'} ) : ( '--check', $opt->{'check'} );
if ( !-f $file ) {
lib/Algorithm/Classifier/IsolationForest/App/Command/proto.pm view on Meta::CPAN
if ( defined $opt->{'o'} ) {
if ($check) {
$self->usage_error('-o may only be used with --from-model');
}
if ( -e $opt->{'o'} && !$opt->{'w'} ) {
$self->usage_error( '-o, "' . $opt->{'o'} . '", already exists and -w is not specified' );
}
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
if ( defined $opt->{'from_model'} ) {
my $model = Algorithm::Classifier::IsolationForest->load( $opt->{'from_model'} );
my $proto_json = $model->to_prototype;
if ( defined $opt->{'o'} ) {
write_file( $opt->{'o'}, { 'atomic' => 1 }, $proto_json . "\n" );
} else {
print $proto_json. "\n";
}
return 1;
} ## end if ( defined $opt->{'from_model'} )
# --check: structural validation plus a human summary of the file.
my $raw = read_file( $opt->{'check'} );
my $proto = eval { Algorithm::Classifier::IsolationForest->validate_prototype($raw) };
die( '--check, "' . $opt->{'check'} . '", is not a valid prototype: ' . $@ ) if $@;
my $schema = $proto->{schema};
my $tags = $schema->{feature_names};
printf " %-20s %s\n", 'file', $opt->{'check'};
printf " %-20s %s\n", 'class', $proto->{class};
printf " %-20s %s\n", 'schema_version', $proto->{schema_version};
printf " %-20s %s\n", 'schema_description', $proto->{schema_description};
printf " %-20s %s\n", 'missing', ( defined $schema->{missing} ? $schema->{missing} : '(unset)' );
printf " %-20s %s\n", 'feature_names', join( ', ', @$tags );
lib/Algorithm/Classifier/IsolationForest/App/Command/set_voting.pm view on Meta::CPAN
contamination carry no threshold and switch without -i.
Switches to new args are like below...
--voting -> voting
-i -> training CSV (contamination models only)
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !-f $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'m'} ) {
$self->usage_error( '-m, "' . $opt->{'m'} . '", is not readable' );
}
if ( !defined( $opt->{'voting'} ) ) {
$self->usage_error('--voting has not been specified');
lib/Algorithm/Classifier/IsolationForest/App/Command/set_voting.pm view on Meta::CPAN
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
}
if ( defined( $opt->{'o'} ) && !$opt->{'w'} && -e $opt->{'o'} ) {
$self->usage_error( '-o, "' . $opt->{'o'} . '", already exists and -w is not specified' );
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
my $iforest = Algorithm::Classifier::IsolationForest->load( $opt->{'m'} );
# A contamination-fitted model needs its training data to relearn the
# threshold for the target mode; without it set_voting would croak, so
# surface the requirement as a friendly usage error pointing at -i. Only
# an actual mode change triggers this -- a no-op switch never recalibrates.
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
--window -> window_size
--eta -> max_leaf_samples
--growth -> growth
--subsample -> subsample
-s -> seed
-c -> contamination
-t -> feature_names
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( !defined( $opt->{'i'} ) ) {
$self->usage_error('-i has not been specified for a file to process');
} elsif ( !-f $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not a file or does not exist' );
} elsif ( !-r $opt->{'i'} ) {
$self->usage_error( '-i, "' . $opt->{'i'} . '", is not readable' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
} elsif ( !-r $opt->{'prototype'} ) {
$self->usage_error( '--prototype, "' . $opt->{'prototype'} . '", is not readable' );
}
if ( defined( $opt->{'t'} ) || defined( $opt->{'mungers'} ) ) {
$self->usage_error(
'--prototype may not be combined with -t or --mungers; the schema comes only from the prototype');
}
} ## end if ( defined( $opt->{'prototype'} ) )
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
# --- resume an existing model first ------------------------------------
# Loaded before the CSV is read because a munger-bearing model changes
# how the CSV is validated (munged columns hold raw values).
my $oif;
if ( -f $opt->{'m'} ) {
$oif = Algorithm::Classifier::IsolationForest->load( $opt->{'m'} );
die( '-m, "' . $opt->{'m'} . '", is not an online model; stream only works on those' . "\n" )
unless ref $oif eq 'Algorithm::Classifier::IsolationForest::Online';
}
# Prototype creation, new models only like the other creation knobs.
# Done before the CSV is read for the same reason resuming is: a
# munger-bearing prototype changes how the CSV is validated. The
# explicit creation switches override the prototype's params.
my $from_proto = 0;
if ( !$oif && defined( $opt->{'prototype'} ) ) {
my $proto = eval {
Algorithm::Classifier::IsolationForest->validate_prototype( scalar read_file( $opt->{'prototype'} ) );
};
die( '--prototype, "' . $opt->{'prototype'} . '", is not a valid prototype: ' . $@ ) if $@;
die( '--prototype, "' . $opt->{'prototype'} . '", is for a batch model; use `iforest fit`' . "\n" )
unless $proto->{class} eq 'online';
my %overrides;
$overrides{'n_trees'} = $opt->{'n'} if defined $opt->{'n'};
$overrides{'window_size'} = $opt->{'window'} if defined $opt->{'window'};
$overrides{'max_leaf_samples'} = $opt->{'eta'} if defined $opt->{'eta'};
$overrides{'growth'} = $opt->{'growth'} if defined $opt->{'growth'};
lib/Algorithm/Classifier/IsolationForest/App/Command/streamc.pm view on Meta::CPAN
--relearn-threshold and renders the reply as text (--json for the raw
reply). The exit code is 0 on ok and non-zero on error, connect
failure, or timeout, so `iforest streamc --set web --ping` works
directly in health checks.
--set/--socket resolve the socket path exactly as streamd does, so the
same flags reach the same daemon.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
if ( defined( $opt->{'set'} ) && $opt->{'set'} !~ /\A[A-Za-z0-9+\-@_]+\z/ ) {
$self->usage_error( '--set, "'
. $opt->{'set'}
. '", must match /\A[A-Za-z0-9+\-@_]+\z/ (letters, digits, and + - @ _ only)' );
}
my @cmds = grep { $opt->{$_} } qw(ping stats save relearn_threshold);
if ( defined( $opt->{'i'} ) ) {
lib/Algorithm/Classifier/IsolationForest/App/Command/streamc.pm view on Meta::CPAN
if ( $opt->{'batch'} < 1 ) {
$self->usage_error( '--batch, "' . $opt->{'batch'} . '", must be >= 1' );
}
if ( $opt->{'timeout'} < 1 ) {
$self->usage_error( '--timeout, "' . $opt->{'timeout'} . '", must be >= 1' );
}
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
# Lazily required for the same reason streamd does it: App::Cmd loads
# every command module up front, and the rest of the CLI should work
# on a box without JSON::MaybeXS.
eval { require JSON::MaybeXS; 1 }
or die( 'iforest streamc requires JSON::MaybeXS for its wire protocol; install it: ' . $@ );
$JSON = JSON::MaybeXS->new( utf8 => 1, canonical => 1 );
lib/Algorithm/Classifier/IsolationForest/App/Command/streamd.pm view on Meta::CPAN
Set names must match /\A[A-Za-z0-9+\-@_]+\z/; since the class has no
"." or "/", a set name can only ever create one new path segment.
Everything under --model-dir and the socket/pid directories is created
at startup when missing; when that fails (e.g. running unprivileged
with the /var defaults) the daemon dies immediately, before forking,
naming the directory and the flag to override.
';
} ## end sub description
sub validate {
my ( $self, $opt, $args ) = @_;
# Anchored with \A/\z rather than ^/$ ($ tolerates a trailing newline).
# The class has no '.' or '/', so a set name can only ever create one
# new path segment -- no traversal is expressible.
if ( defined( $opt->{'set'} ) && $opt->{'set'} !~ /\A[A-Za-z0-9+\-@_]+\z/ ) {
$self->usage_error( '--set, "'
. $opt->{'set'}
. '", must match /\A[A-Za-z0-9+\-@_]+\z/ (letters, digits, and + - @ _ only)' );
}
lib/Algorithm/Classifier/IsolationForest/App/Command/streamd.pm view on Meta::CPAN
} elsif ( !-r $opt->{'prototype'} ) {
$self->usage_error( '--prototype, "' . $opt->{'prototype'} . '", is not readable' );
}
if ( defined( $opt->{'t'} ) || defined( $opt->{'mungers'} ) ) {
$self->usage_error(
'--prototype may not be combined with -t or --mungers; the schema comes only from the prototype');
}
} ## end if ( defined( $opt->{'prototype'} ) )
return 1;
} ## end sub validate
sub execute {
my ( $self, $opt, $args ) = @_;
# JSON::MaybeXS is required lazily so a box without it still has a
# working iforest CLI (App::Cmd loads every command module up front).
eval { require JSON::MaybeXS; 1 }
or die( 'iforest streamd requires JSON::MaybeXS for its wire protocol; install it: ' . $@ );
$JSON = JSON::MaybeXS->new( utf8 => 1, canonical => 1, allow_nonref => 0 );
lib/Algorithm/Classifier/IsolationForest/App/Command/streamd.pm view on Meta::CPAN
}
# --- resume or create the model ----------------------------------------
my $latest = File::Spec->catfile( $OPT{'model_dir'}, 'latest.json' );
if ( -e $latest ) {
$OIF = Algorithm::Classifier::IsolationForest->load($latest);
die( '"' . $latest . '" is not an online model; streamd only works on those' . "\n" )
unless ref $OIF eq 'Algorithm::Classifier::IsolationForest::Online';
} elsif ( defined $OPT{'prototype'} ) {
my $proto = eval {
Algorithm::Classifier::IsolationForest->validate_prototype( scalar read_file( $OPT{'prototype'} ) );
};
die( '--prototype, "' . $OPT{'prototype'} . '", is not a valid prototype: ' . $@ ) if $@;
die( '--prototype, "' . $OPT{'prototype'} . '", is for a batch model; streamd needs an online one' . "\n" )
unless $proto->{class} eq 'online';
my %overrides;
$overrides{'n_trees'} = $OPT{'n'} if defined $OPT{'n'};
$overrides{'window_size'} = $OPT{'window'} if defined $OPT{'window'};
$overrides{'max_leaf_samples'} = $OPT{'eta'} if defined $OPT{'eta'};
$overrides{'growth'} = $OPT{'growth'} if defined $OPT{'growth'};
lib/Algorithm/Classifier/IsolationForest/App/Command/streamd.pm view on Meta::CPAN
sub _threshold {
return
defined $OPT{'threshold'} ? $OPT{'threshold'}
: defined $OIF->decision_threshold ? $OIF->decision_threshold
: 0.5;
}
# One row through the model. A JSON object is a tagged row (full munger
# plan, expanding/combining mungers included); a JSON array is positional
# (scalar mungers, like stream CSV input). Either way the final vector
# is validated numeric before it touches the model -- JSON delivers
# typed values, so anything non-numeric left after munging is a caller
# bug worth an explicit error rather than Perl's silent string-to-0.
# Returns the score, or undef in learn mode. Croaks on any problem.
sub _apply_row {
my ( $row, $mode ) = @_;
my $vec;
if ( ref $row eq 'HASH' ) {
$vec = $OIF->tagged_row_to_array( $row, 'streamd' );
} elsif ( ref $row eq 'ARRAY' ) {
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
perls keep extra low bits in the pure-Perl path). The knob changes
speed, never results.
Batch scoring lazily flattens the mutable trees into the same packed
node layout the batch scorer walks -- online trees are axis-only, and
the online per-leaf depth adjustment rides in the slot the batch packer
uses for its own leaf adjustment -- so C<score_samples>, C<predict>,
C<path_lengths>, C<score_predict_samples>, and C<score_predict_split>
all run through the same C (and OpenMP, when linked) tree walk the
parent uses, with identical results to the pure-Perl fallback. Any
C<learn> invalidates the packed snapshot; the next batch-scoring call
repacks once. C<score_learn> never touches the snapshot: it mutates
the trees after every single point, so its rows are scored by walking
the live trees in C instead.
A model needs to have seen at least C<max_leaf_samples> points before
tree structure exists at all; until then every point scores 1.0. Give
the model a warm-up C<learn()> pass before trusting scores or labels.
Models saved by this class carry their own C<format> tag.
C<< Algorithm::Classifier::IsolationForest->load >> recognises it and
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
schema_description => $args{schema_description},
feature_descriptions => $args{feature_descriptions},
_use_c => $use_c,
_use_openmp => $use_openmp,
};
for my $doc (qw(schema_version schema_description)) {
croak "$doc must be a plain string"
if defined $self->{$doc} && ref $self->{$doc};
}
Algorithm::Classifier::IsolationForest::_validate_feature_descriptions( $self->{feature_names},
$self->{feature_descriptions} )
if defined $self->{feature_descriptions};
# Optional Algorithm::ToNumberMunger integration, identical to the
# parent's: compiled eagerly so spec errors surface here; the module
# is only required when a spec is actually given.
if ( defined $args{mungers} ) {
croak "mungers must be a hashref of 'tag => munger spec'"
unless ref $args{mungers} eq 'HASH';
croak "mungers requires feature_names (the munger plan compiles against them)"
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
}
#-------------------------------------------------------------------------------
# Learning.
#-------------------------------------------------------------------------------
# Advance the stream by one (already prepped) row: every tree learns it
# (subject to subsampling), it enters the window, and the oldest point
# beyond the window is forgotten. This is the single choke point through
# which every tree mutation flows, so it is also where the packed C
# scoring snapshot gets invalidated.
#
# With use_c the per-tree learn and eviction loops run inside the
# parent's C backend (online_learn_row_xs / online_unlearn_row_xs),
# mutating the same live trees this file's Perl recursion would. Random
# draws go through the same generator in the same order, so the trees
# built are bit-identical either way (on nvsize == 8 perls) -- use_c
# only changes speed, matching fit()'s guarantee.
sub _learn_row {
my ( $self, $r ) = @_;
my $sub = $self->{subsample};
$self->_invalidate_c_trees;
if ( _HAS_ONLINE_XS && $self->{_use_c} ) {
Algorithm::Classifier::IsolationForest::online_learn_row_xs(
$self->{trees}, $r, $self->{n_features},
$self->{max_leaf_samples},
( $self->{growth} eq 'adaptive' ? 1 : 0 ), $sub
);
} else {
for my $tree ( @{ $self->{trees} } ) {
next if $sub < 1 && rand() >= $sub;
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
# predict_sums_xs, score_predict_*) applies unchanged. The per-tree
# coefficient buffers are empty -- there are no oblique nodes -- and only
# exist because score_all_xs expects them.
#
# score_learn deliberately never uses this path: it mutates the trees
# after every single point, so the snapshot could never be reused and
# repacking per point would cost more than the walks it replaces.
#-------------------------------------------------------------------------------
# Drop the packed snapshot; called on every mutation.
sub _invalidate_c_trees {
delete @{ $_[0] }{qw(_c_nodes _c_coef_idx _c_coef_val)};
return;
}
# Build (or reuse) the packed snapshot. Returns true when the C scoring
# path may be taken, false when the caller must use the pure-Perl walk.
sub _ensure_c_trees {
my ($self) = @_;
return 0 unless $self->{_use_c};
return 1 if $self->{_c_nodes};
t/04-accel-tuning.t view on Meta::CPAN
#
# * IF_NO_C=1 -- skips building the C backend entirely
# * IF_OPT=<level> -- overrides the default -O3
# * IF_ARCH=<value> -- adds -march=<value>
# * IF_NATIVE=1 -- shorthand for IF_ARCH=native, superseded by IF_ARCH
#
# Since these are read once at load time, each combination needs its own
# fresh perl process -- this spawns one per case and inspects
# $Algorithm::Classifier::IsolationForest::{HAS_C,OPT_LEVEL} from its
# output, the same technique t/03-fit-determinism.t uses for
# OMP_NUM_THREADS. Also checks that IF_OPT/IF_ARCH validate their input
# rather than passing it through to a compiler command line unchecked.
use strict;
use warnings;
use Test::More;
use File::Temp qw(tempfile);
use Algorithm::Classifier::IsolationForest;
my $HAS_C = $Algorithm::Classifier::IsolationForest::HAS_C ? 1 : 0;
t/34-missing-values.t view on Meta::CPAN
$holey[$k][0] = undef if $k % 9 == 0; # missing in column 0
$holey[$k][1] = undef if $k % 13 == 0; # missing in column 1
}
# Score-time test points, some with undef columns.
my @test = ( [ 0.3, 0.3 ], [ 6.0, 6.0 ], [ 0.3, undef ], [ undef, 0.5 ], [ undef, undef ], );
# ---------------------------------------------------------------------------
# Constructor validation (backend-independent)
# ---------------------------------------------------------------------------
subtest 'constructor validates missing / impute_with' => sub {
ok( eval { $CLASS->new( missing => 'zero' ); 1 }, "missing => 'zero' accepted" );
ok( eval { $CLASS->new( missing => 'nan' ); 1 }, "missing => 'nan' accepted" );
ok( !eval { $CLASS->new( missing => 'bogus' ); 1 }, 'bad missing rejected' );
like( $@, qr/missing must be one of/, 'bad missing message' );
ok( eval { $CLASS->new( impute_with => 'median' ); 1 }, "impute_with => 'median' accepted" );
ok( !eval { $CLASS->new( impute_with => 'mode' ); 1 }, 'bad impute_with rejected' );
like( $@, qr/impute_with must be/, 'bad impute_with message' );
is( $CLASS->new->{missing}, 'die', 'missing defaults to die' );
}; ## end 'constructor validates missing / impute_with' => sub
for my $be (@BACKENDS) {
my ( $be_name, $USE_C ) = @$be;
# -----------------------------------------------------------------------
# die (default): fatal on undef in training, scoring still tolerates it
# -----------------------------------------------------------------------
subtest "[$be_name] die mode croaks on undef training data" => sub {
my $f = $CLASS->new( n_trees => 50, seed => $SEED, use_c => $USE_C );
ok( !eval { $f->fit( \@holey ); 1 }, 'fit on holey data croaks' );
t/35-online-accel.t view on Meta::CPAN
subtest 'explicit and edge thresholds agree' => sub {
my $m = make_model();
for my $thr ( 0.2, 0.5, 0.9, 1.5 ) { # 1.5 exercises the non-fast-path fallback
my $perl = with_knobs( $m, 0, 0, sub { $m->predict( \@eval, $thr ) } );
my $c = with_knobs( $m, 1, 0, sub { $m->predict( \@eval, $thr ) } );
is_deeply( $c, $perl, "predict labels identical at threshold $thr" );
}
};
subtest 'mutation invalidates the packed snapshot' => sub {
my $m = make_model();
my $before = with_knobs( $m, 1, 0, sub { $m->score_samples( \@eval ) } );
ok( $m->{_c_nodes}, 'C snapshot exists after a C-path scoring call' );
# Learn a drifted cluster; the stream length also forces window
# evictions, so both learn and unlearn mutations are in play.
srand(8);
$m->learn( cluster( 300, 3 ) );
ok( !$m->{_c_nodes}, 'snapshot dropped by learning' );
my $c_after = with_knobs( $m, 1, 0, sub { $m->score_samples( \@eval ) } );
my $perl_after = with_knobs( $m, 0, 0, sub { $m->score_samples( \@eval ) } );
for my $i ( 0 .. $#eval ) {
cmp_ok( $c_after->[$i], '==', $perl_after->[$i], "post-mutation row $i matches fresh pure Perl" );
}
isnt( $c_after->[0], $before->[0], 'and the scores really did move with the drift' );
}; ## end 'mutation invalidates the packed snapshot' => sub
SKIP: {
skip 'OpenMP not linked in', 1
unless $Algorithm::Classifier::IsolationForest::HAS_OPENMP;
subtest 'OpenMP on/off parity' => sub {
my $m = make_model();
my $serial = with_knobs( $m, 1, 0, sub { $m->score_samples( \@eval ) } );
my $omp = with_knobs( $m, 1, 1, sub { $m->score_samples( \@eval ) } );
for my $i ( 0 .. $#eval ) {
t/37-majority-voting.t view on Meta::CPAN
cmp_ok( $score, '>', 0.5, 'tagged outlier row has a majority vote fraction' );
is( $f->predict_tagged($out), 1, 'tagged outlier row predicted anomalous' );
is( $f->predict_tagged($in), 0, 'tagged inlier row predicted normal' );
my $pair = $f->score_predict_sample_tagged($out);
is( abs( $pair->[0] - $score ) < 1e-12 ? 1 : 0, 1, 'tagged pair score matches score_sample_tagged' );
is( $pair->[1], 1, 'tagged pair label matches predict_tagged' );
}; ## end 'tagged single-row helpers work under majority voting' => sub
# ------------------------------------------------------------------------
# CLI: fit --voting must validate the value and store it on the model.
# ------------------------------------------------------------------------
SKIP: {
my $bin = File::Spec->rel2abs('bin/iforest');
skip "bin/iforest not found", 1 unless -x $bin;
subtest 'CLI fit --voting' => sub {
require File::Temp;
my ( $fh, $csv ) = File::Temp::tempfile( SUFFIX => '.csv', UNLINK => 1 );
for my $row (@data) {
print $fh join( ',', @$row ) . "\n";
t/42-prototype.t view on Meta::CPAN
is_deeply( $o->feature_descriptions, { b => 'the b column' }, 'Online: feature_descriptions accessor' );
ok(
!eval {
$online_class->new( feature_names => ['a'], feature_descriptions => { zzz => 'x' } );
1;
},
'Online: stray feature description croaks'
);
}; ## end 'schema metadata knobs on new()' => sub
subtest 'validate_prototype croak matrix' => sub {
my @cases = (
[ 'not json', 'not { json', qr/did not parse as JSON/ ],
[ 'non-object', '[1,2]', qr/expected a JSON object/ ],
[ 'wrong format tag', { %{ proto_online() }, format => 'Nope' }, qr/format/ ],
[ 'future version', { %{ proto_online() }, version => 2 }, qr/newer than this module/ ],
[ 'unknown top-level key', { %{ proto_online() }, bogus => 1 }, qr/unknown top-level key 'bogus'/ ],
[ 'missing class', { %{ proto_online() }, class => undef }, qr/class of 'batch' or 'online'/ ],
[ 'bad class', { %{ proto_online() }, class => 'streaming' }, qr/class of 'batch' or 'online'/ ],
[
'missing schema_version', { %{ proto_online() }, schema_version => undef },
t/42-prototype.t view on Meta::CPAN
],
[
'batch param on an online prototype',
{ %{ proto_online() }, params => { sample_size => 64 } },
qr/unknown key 'sample_size' for a online prototype/
],
);
for my $case (@cases) {
my ( $name, $proto, $re ) = @$case;
ok( !eval { $batch_class->validate_prototype($proto); 1 }, "$name croaks" );
like( $@, $re, "$name error message" );
}
# The happy paths: a hashref and its JSON encoding both validate,
# and the JSON form decodes back to the same structure.
my $ok = $batch_class->validate_prototype( proto_online() );
is( ref $ok, 'HASH', 'valid hashref prototype returns the hashref' );
my $from_json = $batch_class->validate_prototype( JSON::PP->new->encode( proto_online() ) );
is_deeply( $from_json, proto_online(), 'valid JSON string prototype decodes and validates' );
# Munger-bearing prototypes validate structurally without
# Algorithm::ToNumberMunger -- compilation happens at creation.
my $with_mungers = proto_online();
$with_mungers->{schema}{mungers} = { cpu => { munger => 'anything_here' } };
ok(
eval { $batch_class->validate_prototype($with_mungers); 1 },
'munger-bearing prototype validates structurally'
) or diag $@;
}; ## end 'validate_prototype croak matrix' => sub
subtest 'new_from_prototype: creation, dispatch, overrides' => sub {
my $b = $batch_class->new_from_prototype( proto_batch(), seed => 42 );
isa_ok( $b, $batch_class, 'batch prototype creates the batch class' );
is( $b->{n_trees}, 25, 'params applied' );
is( $b->{sample_size}, 64, 'sample_size applied' );
is( $b->{voting}, 'majority', 'voting applied' );
is( $b->{seed}, 42, 'override applied' );
is( $b->schema_version, 'b1', 'schema_version stamped' );
is( $b->schema_description, 'batch variant', 'schema_description stamped' );
t/42-prototype.t view on Meta::CPAN
my $tuned = $batch_class->new_from_prototype( proto_online(), n_trees => 99 );
is( $tuned->{n_trees}, 99, 'override beats the prototype param' );
ok( !eval { $batch_class->new_from_prototype( proto_online(), mungers => {} ); 1 },
'overriding a schema key croaks' );
like( $@, qr/may not be overridden/, 'schema override error says so' );
ok( !eval { $batch_class->new_from_prototype( proto_online(), sample_size => 64 ); 1 },
'unknown override croaks' );
# Bad param VALUES croak too -- new() itself validates them.
my $bad = proto_online();
$bad->{params}{subsample} = 2;
ok( !eval { $batch_class->new_from_prototype($bad); 1 }, 'invalid param value croaks from new()' );
}; ## end 'new_from_prototype: creation, dispatch, overrides' => sub
subtest 'load_prototype and persistence of the schema metadata' => sub {
my $dir = tempdir( CLEANUP => 1 );
my $path = "$dir/proto.json";
open my $fh, '>', $path or die $!;
t/42-prototype.t view on Meta::CPAN
);
}; ## end 'load_prototype and persistence of the schema metadata' => sub
subtest 'to_prototype extraction round trip' => sub {
my $data = do { srand(13); rows(150) };
my $a = $batch_class->new_from_prototype( proto_batch(), seed => 42, contamination => 0.1 );
$a->fit($data);
my $extracted = $a->to_prototype;
my $decoded = $batch_class->validate_prototype($extracted);
is( $decoded->{class}, 'batch', 'extracted prototype is valid and batch' );
is( $decoded->{schema_version}, 'b1', 'extracted prototype keeps the schema_version' );
is( $decoded->{params}{contamination}, 0.1, 'override made it into the extracted params' );
# Recreate from the extraction with the same seed: identical model.
my $b = $batch_class->new_from_prototype( $extracted, seed => 42 );
$b->fit($data);
is( $b->to_json, $a->to_json, 'model recreated from the extracted prototype is byte-identical' );
# Online extraction round-trips through the validator too.
my $o = $batch_class->new_from_prototype( proto_online() );
my $op = $batch_class->validate_prototype( $o->to_prototype );
is( $op->{class}, 'online', 'online extraction carries class online' );
is( $op->{params}{window_size}, 128, 'online extraction keeps window_size' );
# A model with no feature_names has no variable schema to extract.
my $bare = $batch_class->new;
srand(14);
$bare->fit( rows(80) );
ok( !eval { $bare->to_prototype; 1 }, 'to_prototype without feature_names croaks' );
like( $@, qr/no feature_names/, 'error says why' );
# No recorded metadata -> placeholder values, still a valid file.
my $plain = $batch_class->new( feature_names => [ 'x', 'y' ] );
srand(15);
$plain->fit( rows(80) );
my $placeholder = $batch_class->validate_prototype( $plain->to_prototype );
is( $placeholder->{schema_version}, '0', 'placeholder schema_version when none recorded' );
like(
$placeholder->{schema_description},
qr/none recorded/,
'placeholder schema_description when none recorded'
);
}; ## end 'to_prototype extraction round trip' => sub
subtest 'munger-bearing prototype creation' => sub {
plan skip_all => 'Algorithm::ToNumberMunger is not installed'
( run in 1.603 second using v1.01-cache-2.11-cpan-f4a522933cf )