Algorithm-Classifier-IsolationForest
view release on metacpan or search on metacpan
scoring pass runs through the C backend when use_c is on, so a
learned threshold over a large file is seconds rather than minutes.
The first CSV line is skipped automatically when it holds feature
names (any non-numeric cell, or a match of stored feature_names);
header => 1 still forces it.
- fit_from_csv() now defaults to an "index" gather: a fast block-scan
census records each row's byte offset so the second pass seeks
straight to the sampled rows instead of re-scanning the whole file,
cutting a no-contamination fit of a 2M-row file from ~15s to ~3s.
The offset table costs 8*n bytes and is dropped (falling back to the
streaming reader) once it would exceed index_max (default 256 MiB);
pass index => 0 to force streaming. The contamination scoring pass
defaults to c_scan => 1, letting the C packer coerce cells instead
of validating each in Perl (identical result on valid data; a
non-numeric scored cell becomes 0.0 rather than dying). Under
missing => 'die', a missing cell is now rejected when it lands in a
sampled training row rather than during a full up-front scan.
- fit()/from_json(): _pack_tree, which flattens each tree into the
packed buffers the C scorer walks, now runs in the C backend
(pack_tree_xs) instead of a recursive Perl closure that built an
arrayref and six SVs per node and then flattened the lot through
a map for pack(). It had grown into the largest single phase of
t/data/seeds.sklearn
t/data/wdbc.csv
t/data/wdbc.labels
t/data/wdbc.sklearn
examples/README.md
examples/basic-anomaly-detection.pl
examples/axis-vs-extended.pl
examples/contamination-threshold.pl
examples/save-and-load.pl
examples/server-metrics.pl
examples/online-streaming.pl
benchmarking/bench-sklearn-scoring.pl
benchmarking/bench-fit.pl
benchmarking/bench-score.pl
benchmarking/bench-modes.pl
benchmarking/bench-fit-parallel.pl
benchmarking/bench-extended-fit-accel.pl
benchmarking/bench-axis-fit-accel.pl
benchmarking/bench-extended-predict-accel.pl
benchmarking/bench-axis-predict-accel.pl
benchmarking/BenchAccel.pm
benchmarking/bench-online-score-accel.pl view on Meta::CPAN
$m{c_openmp} = Algorithm::Classifier::IsolationForest::Online->new( %opts, use_c => 1, use_openmp => 1 )
if $HAS_C && $HAS_OPENMP;
for my $name ( sort keys %m ) {
srand(1);
$m{$name}->learn($stream);
}
return \%m;
} ## end sub build_models
print "=" x 70, "\n";
print " online (streaming) scoring accel benchmarks\n";
print " Algorithm::Classifier::IsolationForest::Online\n";
print "=" x 70, "\n";
printf "Backend availability: HAS_C=%d HAS_OPENMP=%d online_learn_xs=%d\n",
$HAS_C, $HAS_OPENMP,
Algorithm::Classifier::IsolationForest::Online::_HAS_ONLINE_XS;
print "(rates shown as calls/second wall-clock; higher is faster)\n";
print "(online_learn_xs=0 means the loaded C object predates the online\n"
. " learn accelerators -- rebuild or rerun with IF_RUNTIME_BUILD=1)\n"
unless Algorithm::Classifier::IsolationForest::Online::_HAS_ONLINE_XS;
examples/README.md view on Meta::CPAN
Each script seeds the RNG so its output is reproducible.
| Script | Shows |
|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `basic-anomaly-detection.pl` | The core workflow: `fit` → `score_samples` → `predict` on a Gaussian blob with known ring outliers, reported as a precision/recall summary and a ranked top-10. |
| `axis-vs-extended.pl` | `mode => 'axis'` vs `mode => 'extended'` on correlated (diagonal) data, illustrating how Extended Isolation Forest reduces the axis-aligned bias and better flags off-diagonal anomalies. |
| `contamination-threshold.pl` | Letting `contamination` auto-learn a cutoff at `fit` time, reading it back with `decision_threshold`, and how `predict` uses it by default vs a naive fixed 0.5. |
| `save-and-load.pl` | Persisting a trained model with `save`/`to_json` and restoring it with `load`/`from_json`, confirming a reloaded model scores bit-for-bit identically. |
| `server-metrics.pl` | An applied take: ranking server requests `[latency_ms, response_bytes]` by anomaly score, using `path_lengths` alongside `score_samples`, and writing the scored data to `request_scores.csv`. |
| `online-streaming.pl` | Online Isolation Forest (`::Online`) on a drifting stream: prequential `score_learn`, and how the sliding window makes the old regime anomalous and the new one normal after a drift. |
## Quick reference
```perl
use Algorithm::Classifier::IsolationForest;
my $if = Algorithm::Classifier::IsolationForest->new(
n_trees => 100, # ensemble size
sample_size => 256, # sub-sample per tree (psi)
examples/online-streaming.pl view on Meta::CPAN
#!/usr/bin/env perl
# online-streaming.pl
#
# Online (streaming) Isolation Forest on a drifting stream. The stream starts
# as a Gaussian blob at the origin, then drifts to a blob at (6, 6). An
# offline model would keep flagging the new regime forever; the online model
# forgets points as they age out of its sliding window, so within one window
# of the drift it treats the new regime as normal and the OLD regime as the
# anomaly.
#
# Points are processed prequentially (score-then-learn), the standard way to
# evaluate a streaming detector: every score reflects the model as it stood
# before that point influenced it.
#
# Run from the distribution root:
# perl -Ilib examples/online-streaming.pl
# or, if the module is installed:
# perl examples/online-streaming.pl
use strict;
use warnings;
use Algorithm::Classifier::IsolationForest::Online;
use constant PI => 3.14159265358979;
srand(7); # reproducible data; the forest gets its own seed below
sub gaussian {
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
force the first line to be skipped even when it is all-numeric.
Options:
=over 4
=item * C<< header => 1 >> -- always skip the first line as a column header
(otherwise a header is auto-detected as described above).
=item * C<< index => 0 >> -- disable the offset-index pass and use the
streaming two-pass reader instead. The index is on by default: it makes
gather a random-access read of just the sampled rows rather than a second
full scan, but costs an C<8 * n>-byte offset table. When that table would
exceed C<index_max> it is dropped automatically and the fit falls back to
streaming. (In index mode a blank line is one that is empty or contains only
whitespace, judged cheaply; pass C<< index => 0 >> for files where that
matters.)
=item * C<< index_max => bytes >> -- ceiling on the offset table (default
256 MiB). Above it, the index is abandoned mid-scan and gather streams.
=item * C<< c_scan => 0 >> -- validate each scored cell with
C<looks_like_number> in the threshold pass instead of letting the C packer
coerce it. C<c_scan> is on by default and takes effect only on the C
mean-voting scoring path, where it is a large saving over millions of rows;
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# ---- Pass 1: census. Establish the true row count (which fixes psi) and
# pin the feature width. Neither census variant parses cells into numbers
# or checks for missing values -- only the rows that actually train
# (_numify_row, in gather) or get scored (the threshold pass) are validated,
# so a malformed or missing cell in a never-sampled row is not seen.
#
# By default the census is an "index" pass: a block-scan that also records
# each data row's byte offset, letting Pass 2 seek straight to the sampled
# rows instead of re-scanning the whole file. The offset table costs 8*n
# bytes; when that would top index_max (or index => 0 was passed) it is not
# built and both passes fall back to the streaming reader.
my $use_index = exists $opt{index} ? $opt{index} : 1;
my $index_max = defined $opt{index_max} ? $opt{index_max} : ( 256 * 1024 * 1024 );
my ( $n, $n_features, $offsets );
if ($use_index) {
( $n, $n_features, $offsets ) = $self->_index_pass( $path, $skip_first, $index_max );
} else {
( $n, $n_features ) = $self->_census_stream( $path, $skip_first );
}
croak "fit_from_csv(): no data rows in '$path'" unless $n;
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# identical to fit()'s per-tree _build_tree(_subsample(...)).
my @trees;
for my $t ( 0 .. $self->{n_trees} - 1 ) {
my $sample = [ @{$pool}{ @{ $tree_idx[$t] } } ];
push @trees, $self->_build_tree( $sample, 0, $limit );
}
$self->{trees} = \@trees;
# Repack the just-built trees for the C scorer up front. fit() scores its
# learned threshold pure-Perl because its training set is small; the
# streaming contamination pass here may score millions of rows, so paying
# one repack now lets that pass run through the C path instead (bit-
# identical result, minutes -> seconds). The delete first clears any stale
# buffers from a prior fit so the repack reflects the new forest.
delete @$self{qw(_c_nodes _c_coef_idx _c_coef_val)};
$self->_rebuild_c_trees() if $self->{_use_c};
# c_scan (default on): let the C scorer's packer coerce raw cells via SvNV
# in the contamination pass instead of validating each with looks_like_number
# in Perl -- a large saving over millions of rows. It only takes effect on
# the C mean-voting path; see _stream_scores.
my $c_scan = exists $opt{c_scan} ? $opt{c_scan} : 1;
$self->_learn_contamination_threshold_streaming( $path, $skip_first, $n, $c_scan )
if defined $self->{contamination};
return $self;
} ## end sub fit_from_csv
=head2 pack_data(\@data)
Returns an opaque, blessed wrapper around the input dataset that the
scoring methods can use directly, skipping the per-call work of walking
the arrayref-of-arrayrefs and converting each cell into a double. At
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
Yousra Chabchoub, Maurras Ulbricht Togbe, Aliou Boly, Raja Chiky (2022). An In-Depth Study and Improvement of Isolation Forest. IEEE Access, vol. 10, 10219 - 10237. 10.1109/ACCESS.2022.3144425 (the Majority Voting Isolation Forest implemented by C<< ...
L<https://ieeexplore.ieee.org/document/9684896>
Mattia Carletti, Matteo Terzi, Gian Antonio Susto (2023). Interpretable Anomaly Detection with DIFFI: Depth-based feature importance of Isolation Forest. Engineering Applications of Artificial Intelligence, vol. 119. 10.1016/j.engappai.2022.105730 (t...
L<https://arxiv.org/abs/2007.11117>
L<https://www.sciencedirect.com/science/article/pii/S0952197622007205>
Filippo Leveni, Guilherme Weigert Cassales, Bernhard Pfahringer, Albert Bifet, Giacomo Boracchi (2024). Online Isolation Forest. (the streaming variant implemented by L<Algorithm::Classifier::IsolationForest::Online>)
L<https://arxiv.org/abs/2505.09593>
L<https://github.com/ineveLoppiliF/Online-Isolation-Forest>
L<https://proceedings.mlr.press/v235/leveni24a.html>
=head1 AUTHOR
Zane C. Bowers-Hadley, C<< <vvelox at vvelox.net> >>
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# (up to libm's own double-vs-long-double disagreements on rare
# rounding ties).
my $s = _to_double( sqrt( -2.0 * _to_double( log($u1) ) ) );
my $c = _to_double( cos( _to_double( TWO_PI * $u2 ) ) );
return _to_double( $s * $c );
} ## end sub _randn
#-------------------------------------------------------------------------------
# Resolve the derived per-fit geometry from the sample count and feature width,
# storing it on the object and returning ($psi, $limit) for the build loop.
# Factored out of fit() so fit_from_csv() -- which learns n from a streaming
# census rather than an in-RAM array -- produces byte-identical psi/extension/
# depth values. Pure arithmetic: consumes no randomness.
#
# Args:
# $n :: total training rows available, a positive integer. From
# scalar @$data in fit(), or the census count in fit_from_csv().
# $n_features :: the feature width, a positive integer.
#
# Returns: the two-element list ($psi, $limit) -- the per-tree sub-sample
# size and the tree height limit. Also sets c_psi, psi_used,
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
$i++;
}
return \%pool;
} ## end sub _gather_stream
# Index census: one block-scan that counts the data rows, pins the feature
# width, AND records each data row's byte offset so gather can seek straight to
# the sampled rows. Blank lines are skipped and the header dropped exactly as
# the reader does, so offset i is the i-th data row. The offset table costs
# 8*n bytes; once it would exceed $max_off it is dropped (returning undef) and
# only the count survives, so the caller falls back to the streaming gather.
#
# Args:
# $path :: path to the CSV file, which must be readable.
# $skip_first :: true to drop the first non-blank line as a header.
# $max_off :: byte ceiling for the offset table. 0 disables the table
# outright, making this a pure counting pass.
#
# Returns: the three-element list ($n, $nf, $offsets) -- the data row count,
# the feature width, and either an arrayref of per-row byte offsets (element
# i is data row i's start) or undef when the table was disabled or went over
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# $path :: path to the CSV file the model was just fitted from.
# $skip_first :: true to drop the first non-blank line as a header.
# $n :: the census row count, which fixes how many rows k stands for.
# $c_scan :: true to let the C packer coerce cells, skipping the Perl
# numeric validation on the scoring passes. See _stream_scores.
#
# Returns: nothing. Sets $self->{threshold} to the same value the in-RAM
# _learn_contamination_threshold would have produced for this data.
#
# Example:
# $self->_learn_contamination_threshold_streaming( 'train.csv', 1, 1_000_000, 1 );
sub _learn_contamination_threshold_streaming {
my ( $self, $path, $skip_first, $n, $c_scan ) = @_;
my $k = int( $self->{contamination} * $n + 0.5 );
$k = 1 if $k < 1;
$k = $n if $k > $n;
# Whole set flagged: sit the cut just below the global minimum score.
if ( $k >= $n ) {
my $min;
$self->_stream_scores( $path, $skip_first, sub { $min = $_[0] if !defined $min || $_[0] < $min }, $c_scan );
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
my $i = $cnt_gt; # first rank holding $v
my $j = $cnt_gt + $cnt_eq; # first rank below $v
if ( $i > 0 && ( $k - $i ) < ( $j - $k ) ) {
$self->{threshold} = ( $above + $v ) / 2.0; # exclude the block
} elsif ( $j < $n ) {
$self->{threshold} = ( $v + $below ) / 2.0; # include the block
} else {
$self->{threshold} = $v - 1e-9; # block runs to the end
}
return;
} ## end sub _learn_contamination_threshold_streaming
# Stream the CSV, score rows in flat batches (through the same mean/majority
# path score_samples/predict use), and invoke $cb->($score) for each row.
# c_scan fast path: when scoring runs through the C packer (mean voting, use_c,
# trees packed) the packer coerces each cell via SvNV, so the reader can run
# raw (parse => 0) and skip the Perl looks_like_number validation. Every other
# path parses (parse => 1) so its Perl scorer sees real numbers.
#
# Args:
# $path :: path to the CSV file, which must be readable.
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
'learn-only',
'Only learn the input (warm-up); no scores are emitted. May not be combined with --score-only.'
],
[
'score-only',
'Only score the input against the model as-is; nothing is learned. May not be combined with --learn-only.'
],
[ 'threshold=f', 'Alternative decision threshold to use for the label column. 0 < $val < 1' ],
[
'save!',
'Save the updated model state back to -m after streaming (default on; --no-save to discard).',
{ 'default' => 1 }
],
# creation knobs, used only when -m does not exist yet
[ 'n=i', 'Number of isolation trees in the ensemble (new models only).' ],
[ 'window=i', 'Sliding window size; 0 disables forgetting (new models only).' ],
[ 'eta=i', 'max_leaf_samples: points a leaf accumulates before splitting (new models only).' ],
[ 'growth=s', "Leaf split-requirement growth, 'adaptive' or 'fixed' (new models only)." ],
[ 'subsample=f', 'Per-tree stream subsampling probability, in (0, 1] (new models only).' ],
[ 's=i', 'Seed int (new models only).' ],
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
'c=f',
'Contamination. Expected fraction of anomalies, in (0, 0.5]; learns the decision threshold from the window (new models only).'
],
[
't=s@',
'Feature name tag. Pass once per feature (e.g. -t cpu -t mem -t disk); the count must match the number of CSV columns or the command will die (new models only).'
],
[
'mungers=s',
'JSON file of Algorithm::ToNumberMunger specs, keyed by feature tag (new models only; requires -t). '
. 'Munged CSV columns may hold raw values; rows are munged before streaming and the spec is '
. 'saved with the model, so resumed runs munge identically. Scalar mungers only for CSV input.',
{ 'completion' => 'files' }
],
[
'prototype=s',
'JSON prototype file to create the model from (new models only): the variable schema and '
. 'schema_version/schema_description come from it, and its params supply knob defaults that the '
. 'creation switches override. May not be combined with -t or --mungers. See PROTOTYPES in the '
. 'module POD.',
{ 'completion' => 'files' }
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
# trusts a flag-matched prebuilt object without inspecting its symbol
# set). Probe once at load: without them, use_c still accelerates the
# packed-snapshot batch scoring -- those functions have been in the
# object all along -- and learning quietly stays pure Perl instead of
# crashing on an undefined XS sub. Rebuilding/reinstalling (or
# IF_RUNTIME_BUILD=1) restores the full set.
use constant _HAS_ONLINE_XS => defined &Algorithm::Classifier::IsolationForest::online_learn_row_xs ? 1 : 0;
=head1 NAME
Algorithm::Classifier::IsolationForest::Online - Online (streaming) Isolation Forest anomaly detection
=head1 SYNOPSIS
use Algorithm::Classifier::IsolationForest::Online;
my $oif = Algorithm::Classifier::IsolationForest::Online->new(
n_trees => 100,
window_size => 2048,
max_leaf_samples => 32,
seed => 42,
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
# persistence keeps the window, so a reloaded model keeps forgetting
# correctly as the stream continues
$oif->save('oiforest_model.json');
my $resumed = Algorithm::Classifier::IsolationForest::Online->load('oiforest_model.json');
=head1 DESCRIPTION
Implements Online Isolation Forest (Online-iForest; Leveni, Weigert
Cassales, Pfahringer, Bifet & Boracchi 2024 -- see REFERENCES), a
streaming variant of Isolation Forest for data that arrives continuously
and whose distribution may drift. There is no C<fit()>: the model
C<learn>s points as they arrive and, once more than C<window_size> points
have been seen, forgets the oldest point for every new one so the model
always reflects the most recent C<window_size> points of the stream.
Trees never store data points. Each node keeps only a running count of
the points that passed through it and the bounding box of their feature
values. A leaf splits once enough points have accumulated (see
C<max_leaf_samples> and C<growth>); because the actual points are gone,
the split simulates them by sampling uniformly inside the leaf's bounding
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
push @rows, $self->tagged_row_to_array( $row->[$i], "learn_tagged (row $i)" );
}
return $self->learn( \@rows );
}
my $vec = $self->tagged_row_to_array( $row, 'learn_tagged' );
return $self->learn( [$vec] );
} ## end sub learn_tagged
=head2 score_learn(\@data)
Prequential (test-then-train) operation, the usual way to run a streaming
detector: each sample is scored against the model as it stood I<before>
that sample was learned, then learned. Returns an arrayref of anomaly
scores, one per sample, in input order.
Unlike the pure scoring methods this works on a brand-new model too (the
first points of a stream simply score 1.0, as nothing is known yet).
my $scores = $oif->score_learn(\@rows);
=cut
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
sub score_sample_tagged {
my ( $self, $row ) = @_;
my $vec = $self->tagged_row_to_array( $row, 'score_sample_tagged' );
my $result = $self->score_samples( [$vec] );
return $result->[0];
}
=head2 explain_samples(\@data, %opts)
Explains, per sample, which features drove its anomaly score, without
learning anything -- the streaming counterpart of the parent class's
method of the same name, returning the identical structure (see
C<explain_samples> in L<Algorithm::Classifier::IsolationForest> for the
full description of the output shape and the C<method> option):
my $explanations = $oif->explain_samples(\@data);
my $top = $explanations->[0]{features}[0];
Differences from the batch class:
The default C<ablation> method substitutes per-feature medians of the
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
sub explain_sample_tagged {
my ( $self, $row, %opts ) = @_;
my $vec = $self->tagged_row_to_array( $row, 'explain_sample_tagged' );
return $self->explain_samples( [$vec], %opts )->[0];
}
=head2 path_lengths(\@data)
Returns an arrayref of the mean isolation depth per sample across the
trees, for inspection -- the streaming counterpart of the parent class's
method of the same name. Depths include the per-leaf count adjustment.
my $depths = $oif->path_lengths(\@data);
=cut
sub path_lengths {
my ( $self, $data ) = @_;
$self->_check_learned;
croak "path_lengths() expects an arrayref of samples"
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
}
}
return ( $lo, $hi );
} ## end sub _box_of
#-------------------------------------------------------------------------------
# Scoring.
#-------------------------------------------------------------------------------
# Depth of the leaf $x lands in, plus the leaf's own depth budget -- the
# streaming analogue of the batch scorer's c(leaf size) adjustment.
# Scoring tolerates undef cells (mapped to 0), matching the parent class.
#
# Args:
# $x :: one sample, an arrayref of feature values. undef cells are
# allowed and count as 0.
# $node :: the node to start walking from, normally a tree's root. Must
# be defined -- callers skip trees that have not grown one.
#
# Returns: the path length as a float: edges walked plus the leaf's
# _rpl(count), so it is rarely a whole number.
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
score => $score,
method => 'ablation',
features => Algorithm::Classifier::IsolationForest::_deltas_to_features(
$names, \@deltas, $data->[$i], $baselines
),
};
} ## end for my $i ( 0 .. $#$data )
return \@out;
} ## end sub _explain_ablation
# Per-feature medians of the retained window -- the streaming
# equivalent of the batch class's fit-time baselines, and better in one
# way: they track drift for free because the window does. Recomputed
# per explanation call (the window moves with the stream); a sort per
# feature over at most window_size values, dwarfed by the scoring batch
# it feeds. Window rows are always dense (missing => die/zero), so no
# undef handling is needed. Without a retained window there is nothing
# to take a median of.
#
# Args: none beyond the model itself.
#
t/21-fit-from-csv.t view on Meta::CPAN
$a->fit_from_csv($csv);
my $b = $CLASS->new(%args);
$b->fit_from_csv($csv);
is( $a->to_json, $b->to_json, 'identical model across two runs' );
}; ## end "[$be_name] deterministic given seed" => sub
subtest "[$be_name] contamination threshold is exact" => sub {
my ($rows) = make_dataset();
my $csv = write_csv($rows);
# Against the very same forest, the streaming learner must land on the
# identical cut the batch learner computes over all rows.
for my $cont ( 0.01, 0.05, 0.1, 0.25, 0.5 ) {
my $m = $CLASS->new(
n_trees => 60,
sample_size => 128,
seed => 5,
contamination => $cont,
use_c => $USE_C,
);
$m->fit_from_csv($csv);
t/21-fit-from-csv.t view on Meta::CPAN
delete @$m{qw(_c_nodes _c_coef_idx _c_coef_val)};
$m->_learn_contamination_threshold($rows);
my $batch = $m->decision_threshold;
ok( abs( $streamed - $batch ) < 1e-12, "contamination=$cont matches batch learner" );
} ## end for my $cont ( 0.01, 0.05, 0.1, 0.25, 0.5 )
}; ## end "[$be_name] contamination threshold is exact" => sub
subtest "[$be_name] tied scores across the boundary" => sub {
# Identical rows produce identical scores, forcing a tie block straddling
# the contamination rank -- the streaming learner's rare second pass.
my @rows = ( ( [ 1, 1 ] ) x 40, map { [ 5 + $_ / 10, 5 + $_ / 10 ] } 1 .. 10 );
my $csv = write_csv( \@rows );
my $m = $CLASS->new(
n_trees => 40,
sample_size => 64,
seed => 9,
contamination => 0.2,
use_c => $USE_C,
);
$m->fit_from_csv($csv);
t/21-fit-from-csv.t view on Meta::CPAN
n_trees => 3,
sample_size => 4,
seed => 2,
contamination => 0.1,
use_c => 1,
);
is( exception { $fast->fit_from_csv($csv) }, undef, 'default c_scan coerces the scored junk row' );
} ## end if ($USE_C)
}; ## end "[$be_name] census defers numeric validation" => sub
subtest "[$be_name] index and streaming paths agree" => sub {
# The default offset-index gather must produce the exact same model as
# the streaming two-pass reader, and as the memory-guarded fallback.
my ($rows) = make_dataset();
my $csv = write_csv($rows);
my %args = (
n_trees => 40,
sample_size => 128,
seed => 7,
contamination => 0.05,
use_c => $USE_C,
);
my $index = $CLASS->new(%args);
$index->fit_from_csv( $csv, index => 1 );
my $stream = $CLASS->new(%args);
$stream->fit_from_csv( $csv, index => 0 );
is( $index->to_json, $stream->to_json, 'index path == streaming path' );
# index_max => 1 forces the offset table over budget on the first row,
# so the fit falls back to streaming -- same model again.
my $fallback = $CLASS->new(%args);
$fallback->fit_from_csv( $csv, index_max => 1 );
is( $fallback->to_json, $stream->to_json, 'over-budget index falls back to streaming' );
}; ## end "[$be_name] index and streaming paths agree" => sub
subtest "[$be_name] c_scan agrees with validated scan on clean data" => sub {
# On valid numbers the C-coerced threshold pass (c_scan => 1) must reach
# the identical model as the Perl-validated pass (c_scan => 0).
my ($rows) = make_dataset();
my $csv = write_csv($rows);
my %args = (
n_trees => 40,
sample_size => 128,
seed => 3,
t/42-prototype.t view on Meta::CPAN
}; ## end 'schema metadata knobs on new()' => sub
subtest 'validate_prototype croak matrix' => sub {
my @cases = (
[ 'not json', 'not { json', qr/did not parse as JSON/ ],
[ 'non-object', '[1,2]', qr/expected a JSON object/ ],
[ 'wrong format tag', { %{ proto_online() }, format => 'Nope' }, qr/format/ ],
[ 'future version', { %{ proto_online() }, version => 2 }, qr/newer than this module/ ],
[ 'unknown top-level key', { %{ proto_online() }, bogus => 1 }, qr/unknown top-level key 'bogus'/ ],
[ 'missing class', { %{ proto_online() }, class => undef }, qr/class of 'batch' or 'online'/ ],
[ 'bad class', { %{ proto_online() }, class => 'streaming' }, qr/class of 'batch' or 'online'/ ],
[
'missing schema_version', { %{ proto_online() }, schema_version => undef },
qr/non-empty schema_version/
],
[ 'empty schema_version', { %{ proto_online() }, schema_version => '' }, qr/non-empty schema_version/ ],
[
'missing schema_description',
{ %{ proto_online() }, schema_description => undef },
qr/non-empty schema_description/
],
t/80-sklearn-comparison-online.t view on Meta::CPAN
}
}
unless ( defined $python_bin ) {
plan skip_all => 'Python with scikit-learn is not installed; skipping cross-language comparison';
}
# -----------------------------------------------------------------------
# Python helper: one batch sklearn IsolationForest per dataset, JSON out.
# Identical to the batch test's helper (sklearn is the fixed reference the
# streaming model converges toward; it is fit once on the full dataset).
#
# sklearn score_samples convention: lower score = more anomalous -- the
# opposite direction from this module, so scores are negated before rank
# correlation.
# -----------------------------------------------------------------------
my $py_script = <<'END_PY';
import sys, json
import numpy as np
from sklearn.ensemble import IsolationForest
( run in 3.954 seconds using v1.01-cache-2.11-cpan-9789f410c06 )