Algorithm-Classifier-IsolationForest
view release on metacpan or search on metacpan
t/02-accel-selection.t
t/03-fit-determinism.t
t/04-accel-tuning.t
t/05-prebuilt-env.t
examples/README.md
examples/basic-anomaly-detection.pl
examples/axis-vs-extended.pl
examples/contamination-threshold.pl
examples/save-and-load.pl
examples/server-metrics.pl
examples/online-streaming.pl
benchmarking/bench-sklearn-scoring.pl
benchmarking/bench-fit.pl
benchmarking/bench-score.pl
benchmarking/bench-modes.pl
benchmarking/bench-fit-parallel.pl
benchmarking/bench-extended-fit-accel.pl
benchmarking/bench-axis-fit-accel.pl
benchmarking/bench-extended-predict-accel.pl
benchmarking/bench-axis-predict-accel.pl
benchmarking/BenchAccel.pm
benchmarking/bench-online-score-accel.pl view on Meta::CPAN
$m{c_openmp} = Algorithm::Classifier::IsolationForest::Online->new( %opts, use_c => 1, use_openmp => 1 )
if $HAS_C && $HAS_OPENMP;
for my $name ( sort keys %m ) {
srand(1);
$m{$name}->learn($stream);
}
return \%m;
} ## end sub build_models
print "=" x 70, "\n";
print " online (streaming) scoring accel benchmarks\n";
print " Algorithm::Classifier::IsolationForest::Online\n";
print "=" x 70, "\n";
printf "Backend availability: HAS_C=%d HAS_OPENMP=%d online_learn_xs=%d\n",
$HAS_C, $HAS_OPENMP,
Algorithm::Classifier::IsolationForest::Online::_HAS_ONLINE_XS;
print "(rates shown as calls/second wall-clock; higher is faster)\n";
print "(online_learn_xs=0 means the loaded C object predates the online\n"
. " learn accelerators -- rebuild or rerun with IF_RUNTIME_BUILD=1)\n"
unless Algorithm::Classifier::IsolationForest::Online::_HAS_ONLINE_XS;
examples/README.md view on Meta::CPAN
Each script seeds the RNG so its output is reproducible.
| Script | Shows |
|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| `basic-anomaly-detection.pl` | The core workflow: `fit` → `score_samples` → `predict` on a Gaussian blob with known ring outliers, reported as a precision/recall summary and a ranked top-10. |
| `axis-vs-extended.pl` | `mode => 'axis'` vs `mode => 'extended'` on correlated (diagonal) data, illustrating how Extended Isolation Forest reduces the axis-aligned bias and better flags off-diagonal anomalies. |
| `contamination-threshold.pl` | Letting `contamination` auto-learn a cutoff at `fit` time, reading it back with `decision_threshold`, and how `predict` uses it by default vs a naive fixed 0.5. |
| `save-and-load.pl` | Persisting a trained model with `save`/`to_json` and restoring it with `load`/`from_json`, confirming a reloaded model scores bit-for-bit identically. |
| `server-metrics.pl` | An applied take: ranking server requests `[latency_ms, response_bytes]` by anomaly score, using `path_lengths` alongside `score_samples`, and writing the scored data to `request_scores.csv`. |
| `online-streaming.pl` | Online Isolation Forest (`::Online`) on a drifting stream: prequential `score_learn`, and how the sliding window makes the old regime anomalous and the new one normal after a drift. |
## Quick reference
```perl
use Algorithm::Classifier::IsolationForest;
my $if = Algorithm::Classifier::IsolationForest->new(
n_trees => 100, # ensemble size
sample_size => 256, # sub-sample per tree (psi)
examples/online-streaming.pl view on Meta::CPAN
#!/usr/bin/env perl
# online-streaming.pl
#
# Online (streaming) Isolation Forest on a drifting stream. The stream starts
# as a Gaussian blob at the origin, then drifts to a blob at (6, 6). An
# offline model would keep flagging the new regime forever; the online model
# forgets points as they age out of its sliding window, so within one window
# of the drift it treats the new regime as normal and the OLD regime as the
# anomaly.
#
# Points are processed prequentially (score-then-learn), the standard way to
# evaluate a streaming detector: every score reflects the model as it stood
# before that point influenced it.
#
# Run from the distribution root:
# perl -Ilib examples/online-streaming.pl
# or, if the module is installed:
# perl examples/online-streaming.pl
use strict;
use warnings;
use Algorithm::Classifier::IsolationForest::Online;
use constant PI => 3.14159265358979;
srand(7); # reproducible data; the forest gets its own seed below
sub gaussian {
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
L<https://ieeexplore.ieee.org/abstract/document/4781136>
Sahand Hariri, Matias Carrasco Kind, Robert J. Brunner (2020). Extended Isolation Forest. 1479 - 1489. 10.1109/TKDE.2019.2947676
L<https://ieeexplore.ieee.org/document/8888179>
Yousra Chabchoub, Maurras Ulbricht Togbe, Aliou Boly, Raja Chiky (2022). An In-Depth Study and Improvement of Isolation Forest. IEEE Access, vol. 10, 10219 - 10237. 10.1109/ACCESS.2022.3144425 (the Majority Voting Isolation Forest implemented by C<< ...
L<https://ieeexplore.ieee.org/document/9684896>
Filippo Leveni, Guilherme Weigert Cassales, Bernhard Pfahringer, Albert Bifet, Giacomo Boracchi (2024). Online Isolation Forest. (the streaming variant implemented by L<Algorithm::Classifier::IsolationForest::Online>)
L<https://arxiv.org/abs/2505.09593>
L<https://github.com/ineveLoppiliF/Online-Isolation-Forest>
L<https://proceedings.mlr.press/v235/leveni24a.html>
=cut
###
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
'learn-only',
'Only learn the input (warm-up); no scores are emitted. May not be combined with --score-only.'
],
[
'score-only',
'Only score the input against the model as-is; nothing is learned. May not be combined with --learn-only.'
],
[ 'threshold=f', 'Alternative decision threshold to use for the label column. 0 < $val < 1' ],
[
'save!',
'Save the updated model state back to -m after streaming (default on; --no-save to discard).',
{ 'default' => 1 }
],
# creation knobs, used only when -m does not exist yet
[ 'n=i', 'Number of isolation trees in the ensemble (new models only).' ],
[ 'window=i', 'Sliding window size; 0 disables forgetting (new models only).' ],
[ 'eta=i', 'max_leaf_samples: points a leaf accumulates before splitting (new models only).' ],
[ 'growth=s', "Leaf split-requirement growth, 'adaptive' or 'fixed' (new models only)." ],
[ 'subsample=f', 'Per-tree stream subsampling probability, in (0, 1] (new models only).' ],
[ 's=i', 'Seed int (new models only).' ],
lib/Algorithm/Classifier/IsolationForest/App/Command/stream.pm view on Meta::CPAN
'c=f',
'Contamination. Expected fraction of anomalies, in (0, 0.5]; learns the decision threshold from the window (new models only).'
],
[
't=s@',
'Feature name tag. Pass once per feature (e.g. -t cpu -t mem -t disk); the count must match the number of CSV columns or the command will die (new models only).'
],
[
'mungers=s',
'JSON file of Algorithm::ToNumberMunger specs, keyed by feature tag (new models only; requires -t). '
. 'Munged CSV columns may hold raw values; rows are munged before streaming and the spec is '
. 'saved with the model, so resumed runs munge identically. Scalar mungers only for CSV input.',
{ 'completion' => 'files' }
],
[
'prototype=s',
'JSON prototype file to create the model from (new models only): the variable schema and '
. 'schema_version/schema_description come from it, and its params supply knob defaults that the '
. 'creation switches override. May not be combined with -t or --mungers. See PROTOTYPES in the '
. 'module POD.',
{ 'completion' => 'files' }
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
# trusts a flag-matched prebuilt object without inspecting its symbol
# set). Probe once at load: without them, use_c still accelerates the
# packed-snapshot batch scoring -- those functions have been in the
# object all along -- and learning quietly stays pure Perl instead of
# crashing on an undefined XS sub. Rebuilding/reinstalling (or
# IF_RUNTIME_BUILD=1) restores the full set.
use constant _HAS_ONLINE_XS => defined &Algorithm::Classifier::IsolationForest::online_learn_row_xs ? 1 : 0;
=head1 NAME
Algorithm::Classifier::IsolationForest::Online - Online (streaming) Isolation Forest anomaly detection
=head1 SYNOPSIS
use Algorithm::Classifier::IsolationForest::Online;
my $oif = Algorithm::Classifier::IsolationForest::Online->new(
n_trees => 100,
window_size => 2048,
max_leaf_samples => 32,
seed => 42,
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
# persistence keeps the window, so a reloaded model keeps forgetting
# correctly as the stream continues
$oif->save('oiforest_model.json');
my $resumed = Algorithm::Classifier::IsolationForest::Online->load('oiforest_model.json');
=head1 DESCRIPTION
Implements Online Isolation Forest (Online-iForest; Leveni, Weigert
Cassales, Pfahringer, Bifet & Boracchi 2024 -- see REFERENCES), a
streaming variant of Isolation Forest for data that arrives continuously
and whose distribution may drift. There is no C<fit()>: the model
C<learn>s points as they arrive and, once more than C<window_size> points
have been seen, forgets the oldest point for every new one so the model
always reflects the most recent C<window_size> points of the stream.
Trees never store data points. Each node keeps only a running count of
the points that passed through it and the bounding box of their feature
values. A leaf splits once enough points have accumulated (see
C<max_leaf_samples> and C<growth>); because the actual points are gone,
the split simulates them by sampling uniformly inside the leaf's bounding
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
push @rows, $self->tagged_row_to_array( $row->[$i], "learn_tagged (row $i)" );
}
return $self->learn( \@rows );
}
my $vec = $self->tagged_row_to_array( $row, 'learn_tagged' );
return $self->learn( [$vec] );
} ## end sub learn_tagged
=head2 score_learn(\@data)
Prequential (test-then-train) operation, the usual way to run a streaming
detector: each sample is scored against the model as it stood I<before>
that sample was learned, then learned. Returns an arrayref of anomaly
scores, one per sample, in input order.
Unlike the pure scoring methods this works on a brand-new model too (the
first points of a stream simply score 1.0, as nothing is known yet).
my $scores = $oif->score_learn(\@rows);
=cut
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
sub score_sample_tagged {
my ( $self, $row ) = @_;
my $vec = $self->tagged_row_to_array( $row, 'score_sample_tagged' );
my $result = $self->score_samples( [$vec] );
return $result->[0];
}
=head2 path_lengths(\@data)
Returns an arrayref of the mean isolation depth per sample across the
trees, for inspection -- the streaming counterpart of the parent class's
method of the same name. Depths include the per-leaf count adjustment.
my $depths = $oif->path_lengths(\@data);
=cut
sub path_lengths {
my ( $self, $data ) = @_;
$self->_check_learned;
croak "path_lengths() expects an arrayref of samples"
lib/Algorithm/Classifier/IsolationForest/Online.pm view on Meta::CPAN
}
}
return ( $lo, $hi );
} ## end sub _box_of
#-------------------------------------------------------------------------------
# Scoring.
#-------------------------------------------------------------------------------
# Depth of the leaf $x lands in, plus the leaf's own depth budget -- the
# streaming analogue of the batch scorer's c(leaf size) adjustment.
# Scoring tolerates undef cells (mapped to 0), matching the parent class.
sub _depth_of {
my ( $self, $x, $node ) = @_;
my $depth = 0;
while ( $node->[_N_TYPE] ) {
$node = ( $x->[ $node->[_N_ATTR] ] // 0 ) < $node->[_N_SPLIT] ? $node->[_N_LEFT] : $node->[_N_RIGHT];
$depth++;
}
return $depth + $self->_rpl( $node->[_N_COUNT] );
}
t/42-prototype.t view on Meta::CPAN
}; ## end 'schema metadata knobs on new()' => sub
subtest 'validate_prototype croak matrix' => sub {
my @cases = (
[ 'not json', 'not { json', qr/did not parse as JSON/ ],
[ 'non-object', '[1,2]', qr/expected a JSON object/ ],
[ 'wrong format tag', { %{ proto_online() }, format => 'Nope' }, qr/format/ ],
[ 'future version', { %{ proto_online() }, version => 2 }, qr/newer than this module/ ],
[ 'unknown top-level key', { %{ proto_online() }, bogus => 1 }, qr/unknown top-level key 'bogus'/ ],
[ 'missing class', { %{ proto_online() }, class => undef }, qr/class of 'batch' or 'online'/ ],
[ 'bad class', { %{ proto_online() }, class => 'streaming' }, qr/class of 'batch' or 'online'/ ],
[
'missing schema_version', { %{ proto_online() }, schema_version => undef },
qr/non-empty schema_version/
],
[ 'empty schema_version', { %{ proto_online() }, schema_version => '' }, qr/non-empty schema_version/ ],
[
'missing schema_description',
{ %{ proto_online() }, schema_description => undef },
qr/non-empty schema_description/
],
t/80-sklearn-comparison-online.t view on Meta::CPAN
}
}
unless ( defined $python_bin ) {
plan skip_all => 'Python with scikit-learn is not installed; skipping cross-language comparison';
}
# -----------------------------------------------------------------------
# Python helper: one batch sklearn IsolationForest per dataset, JSON out.
# Identical to the batch test's helper (sklearn is the fixed reference the
# streaming model converges toward; it is fit once on the full dataset).
#
# sklearn score_samples convention: lower score = more anomalous -- the
# opposite direction from this module, so scores are negated before rank
# correlation.
# -----------------------------------------------------------------------
my $py_script = <<'END_PY';
import sys, json
import numpy as np
from sklearn.ensemble import IsolationForest
( run in 1.753 second using v1.01-cache-2.11-cpan-7fcb06a456a )