Algorithm-Classifier-IsolationForest
view release on metacpan or search on metacpan
- adjust how iforest info displays tag info
0.4.0 2026-07-03/22:45
- Implement named features and methods for testing single rows using
tagged data.
- The C backend can be built and installed at install time, meaning
nothing needs built unless changing the opts. See the docs for
more details.
- new IF_NO_OPENMP=1 selects/builds the serial C backend: no
libgomp linkage and no OpenMP runtime in the process at all
(unlike OMP_NUM_THREADS=1, which just caps the thread count);
IF_NO_OPENMP=0 re-enables OpenMP over a serial install default
- new IF_RUNTIME_BUILD=1 ignores the prebuilt object and forces
the classic runtime build even when the flags match
- iforest accel updated to reflect various changes to the C backend
- scoring: the per-leaf path-length adjustment c(size) is now
precomputed at tree-pack time and stored in the (previously
unused) third slot of packed leaf records, removing a log()
call per point per tree from the C scoring hot loop -- about
25% faster axis-mode scoring; results are bit-identical
- scoring: score_all_xs now picks between two loop shapes based
flags actually used are exposed via $OPT_LEVEL
- Benchmarking: bench-sklearn-scoring.pl now compares pure Perl, C,
and C+OpenMP fit/score paths against sklearn side by side
0.2.1 2026-06-30/14:30
- derp... actually update MANIFEST so a bunch of files from last release
are actually included
0.2.0 2026-06-30/14:15
- C acceleration via Inline::C for core fit and predict ops
- OpenMP support for parallel multi-threaded fitting and predict ops
- SIMD (AVX/SSE) acceleration where available
- Data packing support for compact model storage (new `pack` CLI command)
- Parallel fit capability
- New `score_predict_split` method
- New `accel` CLI command for querying available acceleration flags
- New `bench` CLI command for running built-in benchmarks
- New `info` CLI command with expanded model introspection
- Benchmarking scripts covering fit, predict, scoring, and accel modes
- Tests: accel flag detection, accel selection, undef column handling,
data packing, parallel fit, sklearn comparison (including undef),
benchmarking/BenchAccel.pm view on Meta::CPAN
return sprintf '%.2f/s', $r if $r < 100;
return sprintf '%.0f/s', $r;
}
# wall_rate($code, $secs) -- scalar ops/second over $secs wall-clock
# seconds. Returns the rate as a plain number so callers can format
# their own tables.
#
# The measurement is split into 3 equal sub-windows and the median of
# their rates is returned. At small workloads (a few hundred
# microseconds per call) OpenMP thread scheduling is non-deterministic
# enough that a single 2-second window can land in a slow stretch and
# report ~30x lower throughput than steady state; median across
# windows smooths that without spending extra time.
sub wall_rate {
my ( $code, $secs ) = @_;
$secs ||= 1;
# Warm-up: at least 0.3 s (matches the original Perl bench() helper
# this replaced) so OpenMP thread pools are firmly hot before any
# measurement window starts. Scales up for long budgets -- a 30 s
# measurement gets a 3 s warmup.
my $warmup = $secs * 0.1;
$warmup = 0.3 if $warmup < 0.3;
my $wt0 = time;
$code->() while time - $wt0 < $warmup;
my $WINDOWS = 3;
my $win_secs = $secs / $WINDOWS;
my @rates;
benchmarking/bench-axis-predict-accel.pl view on Meta::CPAN
#!/usr/bin/perl
# benchmarking/bench-axis-predict-accel.pl
#
# Benchmarks axis-mode (classic IF) scoring/prediction under each
# acceleration backend:
# pure_perl -- use_c => 0 (pure Perl tree walk)
# c_serial -- use_c => 1, use_openmp => 0 (C tree walk, single thread)
# c_openmp -- use_c => 1, use_openmp => 1 (C tree walk, OpenMP parallel)
#
# This is where the accel differences are largest: the C backend replaces
# the Perl tree walk with a tight C loop, and OpenMP distributes query
# points across cores.
#
# Sections:
# 1. Scoring method comparison -- all 5 methods under each backend
# 2. Query set size scaling -- where OpenMP parallelism shines
# 3. n_trees scaling -- more trees = more work per point
benchmarking/bench-extended-predict-accel.pl view on Meta::CPAN
#!/usr/bin/perl
# benchmarking/bench-extended-predict-accel.pl
#
# Benchmarks extended-mode (EIF) scoring/prediction under each
# acceleration backend:
# pure_perl -- use_c => 0 (pure Perl tree walk)
# c_serial -- use_c => 1, use_openmp => 0 (C tree walk, single thread)
# c_openmp -- use_c => 1, use_openmp => 1 (C tree walk, OpenMP parallel)
#
# Extended mode adds an oblique dot product at every internal node. The
# C backend uses `#pragma omp simd` to auto-vectorize that dot product
# (when OpenMP 4.0+ is available), so the c_serial vs c_openmp gap may
# be wider here than in axis mode, especially at high feature counts.
#
# Sections:
# 1. Scoring method comparison -- all 5 methods under each backend
# 2. Query set size scaling -- where OpenMP parallelism shines
benchmarking/bench-sklearn-scoring.pl view on Meta::CPAN
#!/usr/bin/perl
# benchmarking/bench-sklearn-scoring.pl
#
# Compares scoring throughput among three Perl backends and scikit-learn's
# IsolationForest across two axes:
# * query-set size (fixed feature count)
# * feature-vector width (fixed query-set size)
#
# Perl backends benchmarked (when available):
# pure perl -- no Inline::C (use_c => 0)
# C serial -- Inline::C, single-threaded (use_c => 1, use_openmp => 0)
# C+OpenMP -- Inline::C + OpenMP parallel (use_c => 1, use_openmp => 1)
#
# The same training CSV and query CSVs are used by all sides so the
# comparison is on identical data. Models are pre-trained before any
# timing starts.
#
# Method correspondence:
# Perl score_samples <--> clf.score_samples(X) (same formula, opposite sign)
# Perl predict <--> clf.predict(X) (same semantics, 0/1 vs -1/+1)
# Perl score_predict_samples <--> (no sklearn equivalent)
benchmarking/bench-sklearn-scoring.pl view on Meta::CPAN
my $m = $exp->{pure_model};
$pure_pl{$key} = {
score_samples => wall_rate( sub { $m->score_samples($q) }, $BENCH_SECS ),
predict => wall_rate( sub { $m->predict($q) }, $BENCH_SECS ),
score_predict_samples => wall_rate( sub { $m->score_predict_samples($q) }, $BENCH_SECS ),
score_predict_split => wall_rate( sub { $m->score_predict_split($q) }, $BENCH_SECS ),
path_lengths => wall_rate( sub { $m->path_lengths($q) }, $BENCH_SECS ),
};
}
# C serial (single-threaded Inline::C)
if ($HAS_C) {
my $m = $exp->{c_model};
my $packed = $m->pack_data($q);
$c_pl{$key} = {
score_samples => wall_rate( sub { $m->score_samples($q) }, $BENCH_SECS ),
score_samples_packed => wall_rate( sub { $m->score_samples($packed) }, $BENCH_SECS ),
predict => wall_rate( sub { $m->predict($q) }, $BENCH_SECS ),
predict_packed => wall_rate( sub { $m->predict($packed) }, $BENCH_SECS ),
score_predict_samples => wall_rate( sub { $m->score_predict_samples($q) }, $BENCH_SECS ),
score_predict_split => wall_rate( sub { $m->score_predict_split($q) }, $BENCH_SECS ),
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
double sum = 0.0;
for (int t = 0; t < n_trees; t++) {
sum += if_walk_tree(node_ptrs[t], idx_ptrs[t],
val_ptrs[t], xi, n_feats);
}
sm[i] = sum;
}
}
else {
/* 256 rows x 16 features x 8 bytes = 32 KB of input per block
* -- comfortable in L2 next to one tree. Each OpenMP thread
* owns whole blocks and therefore a unique slice of sm[], so
* there is still no synchronisation. For small batches the
* tile shrinks to keep ~4 blocks per thread available; losing
* per-block tree reuse there is fine, since a small batch
* never re-streams much anyway. */
int tile = 256;
#ifdef _OPENMP
if (use_openmp) {
int min_blocks = omp_get_max_threads() * 4;
if (min_blocks > 0 && (n_pts + tile - 1) / tile < min_blocks) {
tile = (n_pts + min_blocks - 1) / min_blocks;
if (tile < 1) tile = 1;
}
}
#endif
int n_blocks = (n_pts + tile - 1) / tile;
#ifdef _OPENMP
#pragma omp parallel for schedule(static) if(use_openmp)
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
free(all);
}
/* ---------------------------------------------------------------------
* build_forest_openmp_xs -- OpenMP-parallel fit() tree builder.
*
* build_forest_xs (above) is bit-identical to the pure-Perl path
* because every random draw goes through Drand01(), the same
* generator Perl's rand()/srand() use -- but that generator is a
* single mutable struct shared by the whole interpreter, so calling
* it concurrently from multiple OpenMP threads would be a data race.
* The same is true of any Perl API call (newAV, newSViv, ...): Perl's
* SV allocator isn't safe to call from multiple OS threads sharing one
* interpreter without a lock that would just serialise everything
* anyway.
*
* So this builder trades the bit-identical guarantee for real thread
* parallelism: each tree gets its own splitmix64 PRNG stream, seeded
* from a tree index (not thread id or scheduling order), so results
* are still reproducible for a fixed seed and n_trees regardless of
* OMP_NUM_THREADS -- just different from what build_forest_xs or the
* pure-Perl path would produce for the same seed. The one Drand01()
* call in this function happens before the parallel region starts
* (single-threaded), so the result still varies with the model's
* `seed` the way every other code path does; it isn't used inside the
* parallel loop.
*
* Each tree is built entirely with plain C data (row-index int arrays,
* a growable TreeBuf of packed doubles/ints) -- no Perl API call
* happens anywhere inside the parallel region. Each node record in
* TreeBuf uses _pack_tree's 6-double SoA layout (see the file-top
* comment), but the node ORDER differs: records are appended
* post-order (a node is pushed after both its children, since child
* indices must be known first), so the root is the last record --
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
b->cap_val = newcap;
}
memcpy(b->idx + b->n_idx, idx, (size_t)n * sizeof(int));
memcpy(b->val + b->n_val, val, (size_t)n * sizeof(double));
b->n_idx += n;
b->n_val += n;
return off;
}
/* splitmix64 -- fast, well-mixed, and per-stream state fits in one
* uint64_t, which is all a thread-private PRNG needs here. Not
* cryptographic; doesn't need to be. */
static uint64_t sm64_next(uint64_t *s) {
uint64_t z = (*s += 0x9E3779B97F4A7C15ULL);
z = (z ^ (z >> 30)) * 0xBF58476D1CE4E5B9ULL;
z = (z ^ (z >> 27)) * 0x94D049BB133111EBULL;
return z ^ (z >> 31);
}
static double sm64_drand(uint64_t *s) {
return (double)(sm64_next(s) >> 11) * (1.0 / 9007199254740992.0);
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
static double _ts_randn(uint64_t *s) {
double u1 = sm64_drand(s);
double u2;
if (u1 == 0.0) u1 = 1e-12;
u2 = sm64_drand(s);
return sqrt(-2.0 * log(u1)) * cos(6.283185307179586 * u2);
}
/* Thread-safe twin of _build_node_c: same split algorithm, but reads
* randomness from a thread-private splitmix64 stream instead of
* Drand01(), and writes into a TreeBuf instead of allocating Perl AVs
* -- so it touches no interpreter-global state and is safe to call
* concurrently from an OpenMP parallel region, one tree per thread. */
static int _build_node_packed(const double* x, int nf, int* idxs, int size,
int depth, int limit, int mode_flag,
int ext_active, TreeBuf *buf, uint64_t *rng) {
double *lo, *hi;
int *varying, nv, f, my_idx;
if (depth >= limit || size <= 1) {
my_idx = tb_push_node(buf, 0.0, (double)size, 0.0, 0.0, 0.0, 0.0);
free(idxs);
return my_idx;
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
hi = (double*)malloc(nf * sizeof(double));
for (f = 0; f < nf; f++) {
lo[f] = HUGE_VAL;
hi[f] = -HUGE_VAL;
}
for (int i = 0; i < size; i++) {
const double* row = x + (size_t)idxs[i] * (size_t)nf;
/* See the matching comment in _build_node_c: no isnan() guard
* needed, since NaN < x / NaN > x are always false already --
* that's what lets this vectorize as a plain min/max scan.
* omp simd here is thread-safe to call from inside the caller's
* omp parallel region: it's a per-thread vectorization hint,
* not a team construct, so it doesn't nest into anything. */
#ifdef _OPENMP
#pragma omp simd
#endif
for (int f2 = 0; f2 < nf; f2++) {
double v = row[f2];
if (v < lo[f2]) lo[f2] = v;
if (v > hi[f2]) hi[f2] = v;
}
}
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
* still a plain serial call into the interpreter's RNG state. */
base_seed = (uint64_t)(Drand01() * 18446744073709551615.0);
bufs = (TreeBuf*)malloc((size_t)n_trees * sizeof(TreeBuf));
for (t = 0; t < n_trees; t++) tb_init(&bufs[t]);
#ifdef _OPENMP
#pragma omp parallel for schedule(dynamic) if(use_openmp)
#endif
for (int t = 0; t < n_trees; t++) {
/* Seeded from the tree index, not thread id or iteration order,
* so the mapping from tree -> RNG stream is independent of
* OMP_NUM_THREADS / scheduling. sm64_next() mixes once more so
* adjacent tree indices (which differ by one golden-ratio step)
* don't start from too-similar states. */
uint64_t rng = base_seed + (uint64_t)t * 0x9E3779B97F4A7C15ULL;
rng = sm64_next(&rng);
int *all = (int*)malloc((size_t)n_pts * sizeof(int));
int *sample;
int i;
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# split value cascades into a structurally different tree). The
# -march speedup comes from AVX2 vectorization, not contraction,
# so this costs little (verified against the fit-determinism and
# scoring-parity tests).
my $opt_level = $opt;
$opt_level .= " -march=$arch -ffp-contract=off" if length $arch;
# IF_NO_OPENMP=1 forces the serial C build: the OpenMP compile attempt
# is skipped, so the object has no libgomp linkage and never starts an
# OpenMP runtime in the process. Distinct from OMP_NUM_THREADS=1,
# which runs the parallel code on a single thread but still loads
# libgomp. An explicit IF_NO_OPENMP=0 re-enables OpenMP over a
# no-openmp configure-time default.
my $no_omp
= defined $ENV{IF_NO_OPENMP}
? ( $ENV{IF_NO_OPENMP} ? 1 : 0 )
: $def_no_omp;
# The prebuilt object is only trusted when the effective flags match
# what it was compiled with; any difference -- or an explicit
# IF_RUNTIME_BUILD=1 -- falls through to the classic runtime Inline::C
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
random draws go through the same generator C<rand()>/C<srand()> use
internally, in the same call order the pure-Perl builder uses, so a
given C<seed> produces bit-identical trees whether C<use_c> is on or
off -- switching backends changes only how fast the model is built, not
the model itself. On perls whose NV is wider than a C double
(C<-Duselongdouble> / C<-Dusequadmath>) the pure-Perl builder rounds
each stored value to double precision to preserve this parity; axis
mode matches exactly, while extended mode can still differ on rare
libm rounding ties (double vs long-double transcendentals).
By default this C builder is single-threaded per call, because Perl's
RNG state isn't safe to share across OpenMP threads. Two ways to scale
fit() across cores are available (see below for why they don't compose):
=over 4
=item * C<parallel_fit> forks N worker processes, each building its
share of the trees with the (still single-threaded) C builder. Fixed
IPC/serialisation overhead per worker means this can cost more than it
saves once a fit already completes in milliseconds; it's most useful
once a single-process fit is large enough that the fork/Storable
overhead is small relative to the work being split.
=item * C<use_openmp_fit> builds trees across OpenMP threads within a
single process (one tree per thread), using a separate, thread-safe
PRNG seeded per tree index instead of Perl's C<rand()>. This means
trees built with C<use_openmp_fit> are I<not> bit-identical to the
default C<use_c> path for the same seed -- but a fixed seed and
C<n_trees> still reproduce the same trees regardless of
C<OMP_NUM_THREADS> or how OpenMP schedules the work. It's off by
default (unlike C<use_c>/C<use_openmp>, which only ever change speed,
this changes which trees get built) and only takes effect when C<use_c>
is also on and OpenMP is linked in.
=back
These two do NOT compose, despite both existing to parallelise fit().
A process that has run any OpenMP region -- including plain
C<score_samples()>/C<predict()> with the default C<use_openmp> -- and
then C<fork()>s (as C<parallel_fit> does) hands each child a copy of
libgomp's thread pool whose worker threads did not survive the fork. A
child that then starts its own C<#pragma omp parallel> region (as
C<use_openmp_fit> would) tries to reuse that now-invalid pool and
hangs. This is a general limitation of combining C<fork()> with OpenMP,
not something fixable from Perl, so C<parallel_fit>'s forked workers
always use the single-threaded C builder regardless of
C<use_openmp_fit> -- setting both just means C<parallel_fit> wins and
C<use_openmp_fit> has no effect for that call.
Detection happens once when the module is loaded. When the
distribution was installed with C<Inline::C> available, the C backend
was already compiled during C<make> and the installed object is loaded
directly (see L</Compile at install time (the prebuilt object)> below);
otherwise the backend is compiled on first load and the artefact is
cached under C<_Inline/> and reused on subsequent runs. Five package
variables report what the load picked up:
$Algorithm::Classifier::IsolationForest::HAS_C # 0/1
$Algorithm::Classifier::IsolationForest::HAS_OPENMP # 0/1
$Algorithm::Classifier::IsolationForest::HAS_SIMD # 0/1 (OpenMP 4.0+)
$Algorithm::Classifier::IsolationForest::OPT_LEVEL # e.g. "-O3 -march=native", '' if HAS_C is 0
$Algorithm::Classifier::IsolationForest::C_SOURCE # 'prebuilt' / 'runtime', '' if HAS_C is 0
Neither dependency is required. Without C<Inline::C> the module falls
back to a pure-Perl implementation that produces identical results, just
slower; without OpenMP the C backend runs single-threaded.
The bundled C<iforest accel> subcommand performs a tiny fit + score and
prints which backend is active (including the build flags below), which
is the recommended way to verify the build picked up the optional
dependencies on a given machine.
=head2 Compile at install time (the prebuilt object)
When C<Inline::C> is usable while the distribution itself is being
built, C<perl Makefile.PL> arranges for the C backend to be compiled
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
instruction sets the build host happens to have, including AVX-512 on
some Intel CPUs -- which is known to trigger clock throttling under
sustained heavy use and can make throughput I<worse> than a
conservative target like C<x86-64-v3> (AVX2, no AVX-512). If in doubt,
benchmark both before committing to one.
=item * C<IF_NO_OPENMP=1> -- build (or select) the serial C backend: the
OpenMP compile attempt is skipped entirely, so the resulting object has
no libgomp linkage and never starts an OpenMP runtime inside the
process. This differs from C<OMP_NUM_THREADS=1>, which merely runs the
parallel code on one thread but still loads libgomp. Set at
C<perl Makefile.PL> time it yields a serial prebuilt object; set at run
time against an OpenMP prebuilt install it triggers a runtime serial
build (needing a compiler). An explicit C<IF_NO_OPENMP=0> re-enables
OpenMP over a serial configure-time default.
=back
Whichever of these are used, the cached artefact under C<_Inline/> is
pinned to that build's instruction set -- delete C<_Inline/> (or use a
separate one per host) if the directory is shared across machines with
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
=head2 Tuning the OpenMP runtime
These are standard OpenMP environment variables libgomp already reads
at run time (set before running your script, no module-specific
handling needed) -- listed here because they matter most for exactly
the workloads this module has: C<score_all_xs>'s per-point parallel
loop and C<use_openmp_fit>'s per-tree parallel loop.
=over 4
=item * C<OMP_NUM_THREADS=N> -- caps how many threads a parallel region
uses. Useful to leave headroom for other work sharing the machine, or
to pin down C<use_openmp_fit> reproducibility checks (see its docs
above: results don't depend on this, but it's a natural thing to vary
when confirming that).
=item * C<OMP_PROC_BIND=close> / C<OMP_PLACES=cores> -- on multi-socket
or otherwise NUMA machines, pins each thread to a core near where its
data already lives instead of letting the OS scheduler migrate threads
across sockets mid-run. Both C<score_all_xs> (each thread scans its own
slice of the packed query buffer) and C<use_openmp_fit> (each thread
builds one tree from packed training data) benefit from this when the
input is large enough to not fit comfortably in one socket's cache.
=back
These cost nothing to try -- unlike C<IF_ARCH>/C<IF_NATIVE>, they're
read fresh every run, not baked into a cached binary, so there's no
downside to experimenting per invocation.
=head1 GENERAL METHODS
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
- use_c :: boolean, override whether the Inline::C backend is used for
both scoring and fit()'s tree builder. When false the instance
falls back to pure Perl for both even if the C backend compiled
successfully. When true (or unset) the C backend is used if
available ($HAS_C). fit() with use_c on produces bit-identical
trees to use_c off for the same seed -- only build speed differs.
default :: $HAS_C
- use_openmp :: boolean, override whether OpenMP parallel scoring is
used inside score_all_xs(). When false the C tree walk runs
single-threaded even if OpenMP was linked in. Ignored when
use_c is false (pure Perl has no OpenMP path).
default :: $HAS_OPENMP
- use_openmp_fit :: boolean, build fit()'s trees across OpenMP threads
(one tree per thread) instead of the single-threaded C builder.
Opt-in and off by default: unlike use_c/use_openmp, this changes
which trees get built. Perl's RNG isn't safe to call from
multiple OS threads sharing one interpreter, so this path seeds
an independent PRNG per tree from the tree index rather than
Drand01() -- trees differ from the use_c (single-threaded)
and pure-Perl paths even with the same seed, though a fixed
seed and n_trees still reproduce the same trees regardless of
OMP_NUM_THREADS or scheduling. Does NOT compose with
parallel_fit: a forked child starting its own OpenMP region
after the parent process has used OpenMP for anything can
hang (a general fork()+libgomp limitation), so parallel_fit's
workers always use the single-threaded C builder regardless
of this setting -- setting both just means parallel_fit wins.
Ignored (clamped to 0) when use_c is false or OpenMP isn't
linked in.
default :: 0
Note: log2 under Perl is as below...
log($psi) / log(2)
=cut
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
my $nodes_packed = pack( 'd*', map { @$_ } @node_data );
my $idx_packed = @coef_idx ? pack( 'l*', @coef_idx ) : pack('l*');
my $val_packed = @coef_val ? pack( 'd*', @coef_val ) : pack('d*');
return ( $nodes_packed, $idx_packed, $val_packed );
} ## end sub _pack_tree
# Build packed C-ready representations for all trees and store them in
# $self->{_c_nodes}, $self->{_c_coef_idx}, $self->{_c_coef_val}.
# Called after fit() and from_json() when _use_c is true. n_features is
# threaded through so _pack_tree can spot the dense-pack opportunity.
sub _rebuild_c_trees {
my ($self) = @_;
my ( @c_nodes, @c_coef_idx, @c_coef_val );
for my $tree ( @{ $self->{trees} } ) {
my ( $np, $ip, $vp ) = _pack_tree( $tree, $self->{n_features} );
push @c_nodes, $np;
push @c_coef_idx, $ip;
push @c_coef_val, $vp;
}
$self->{_c_nodes} = \@c_nodes;
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
# child
close $rh;
binmode $wh;
if ( defined $self->{seed} ) {
srand( $self->{seed} + $w * 1009 );
}
# Deliberately never _build_forest_openmp here, even when
# use_openmp_fit is on: if this process (or the parent that
# fork()ed us) already ran any OpenMP region before this
# fork -- including plain score_samples()/predict() with
# the default use_openmp -- libgomp's thread pool exists
# but its worker threads didn't survive the fork. A child
# starting its own #pragma omp parallel region then tries
# to reuse that now-invalid pool and hangs. This is a
# general fork()+libgomp limitation, not fixable from here,
# so forked workers always use the single-threaded C
# builder (or pure Perl) instead. See t/03-fit-determinism.t
# and the NATIVE ACCELERATION docs for the observed hang and
# why parallel_fit + use_openmp_fit isn't composed for real.
my $trees;
if ( $self->{_use_c} ) {
$trees = $self->_build_forest_c( $data, $psi, $limit, $share );
} else {
my @t;
for ( 1 .. $share ) {
my $sample = _subsample( $data, $psi );
lib/Algorithm/Classifier/IsolationForest.pm view on Meta::CPAN
my $mode_flag = $self->{mode} eq 'extended' ? 1 : 0;
my $ext_level = $self->{extension_level_used} // 0;
my $trees = [];
build_forest_xs( $x_packed, $n, $nf, $n_trees, $psi, $limit, $mode_flag, $ext_level, $trees );
return $trees;
} ## end sub _build_forest_c
#-------------------------------------------------------------------------------
# OpenMP-parallel fit(): builds $n_trees trees across OpenMP threads (one
# tree per thread) via build_forest_openmp_xs. Unlike _build_forest_c,
# random draws come from a thread-private PRNG seeded per tree index
# rather than Drand01() -- Perl's RNG state can't be shared safely
# across OpenMP threads -- so the resulting trees are NOT bit-identical
# to the use_c (serial) or pure-Perl paths for the same seed, though a
# fixed seed + n_trees still reproduce the same trees regardless of
# OMP_NUM_THREADS. This is why it's gated by the separate, opt-in
# use_openmp_fit knob rather than reusing use_c/use_openmp.
#
# Only called from fit()'s non-forked branch. _fit_trees_parallel's
# workers never call this, even when use_openmp_fit is on: a forked
# child starting its own OpenMP region after the parent process has
# used OpenMP for anything (this includes plain score_samples()) can
# hang -- see the comment above that branch for the fork()+libgomp
t/03-fit-determinism.t view on Meta::CPAN
# Covers:
# 1. Each backend, run twice with the same seed, builds bit-identical
# trees (not just bit-identical scores -- the actual tree structure).
# 2. Pure-Perl and serial-C build BIT-IDENTICAL trees for the same seed,
# in both axis and extended mode, and across every `missing`
# strategy -- because the serial C builder draws randomness through
# Drand01(), the same generator Perl's own rand()/srand() use.
# 3. use_openmp_fit is reproducible for a fixed seed + n_trees
# regardless of OMP_NUM_THREADS (checked across separate
# subprocesses, since that's the only way to force a different
# thread count) -- it deliberately does NOT match the Drand01-based
# backends (documented behaviour: it uses its own thread-safe PRNG),
# so that's checked as a "differs, but each is internally consistent"
# property rather than cross-backend equality.
# 4. use_openmp_fit + parallel_fit together is still safe and
# reproducible -- but does NOT actually run forked workers in
# OpenMP. A forked child starting its own OpenMP region after
# the parent process has used OpenMP for anything (including
# plain score_samples()) can hang -- a general fork()+libgomp
# limitation -- so parallel_fit's workers always use the
# single-threaded C builder regardless of use_openmp_fit. This
# was caught by an earlier version of this test hanging.
use strict;
use warnings;
use Test::More;
use File::Temp qw(tempfile);
use Config;
use JSON::PP ();
use Algorithm::Classifier::IsolationForest;
t/03-fit-determinism.t view on Meta::CPAN
use_c => 1,
use_openmp_fit => 1,
);
is( fit_trees(%opts), fit_trees(%opts), "[$mode] two use_openmp_fit fits with the same seed match" );
} ## end for my $mode (qw(axis extended))
}; ## end 'use_openmp_fit is reproducible in-process for a fixed seed' => sub
subtest 'use_openmp_fit trees are valid, but not required to match Drand01 backends' => sub {
plan skip_all => 'OpenMP not linked in' unless $HAS_C && $HAS_OPENMP;
# use_openmp_fit uses its own thread-safe PRNG (documented: Perl's
# RNG state can't be shared across OpenMP threads), so it is NOT
# expected to match the serial-C/pure-Perl trees for the same seed.
# What matters is that it still produces a valid, usable model.
my $f = $CLASS->new(
n_trees => 40,
sample_size => 48,
seed => 42,
mode => 'extended',
use_c => 1,
use_openmp_fit => 1,
)->fit( \@data );
t/03-fit-determinism.t view on Meta::CPAN
is( $exit4, 0, 'OMP_NUM_THREADS=4 child exits cleanly' )
or diag("child output:\n$out4");
is( $out1, $out4, 'same seed + n_trees builds identical trees under OMP_NUM_THREADS=1 vs 4' );
}; ## end 'use_openmp_fit reproducibility does not depend on OMP_NUM_THREADS' => sub
# ------------------------------------------------------------------------
# 5. use_openmp_fit + parallel_fit together: must not hang, and must
# still be reproducible -- but see the file-top comment: parallel_fit's
# forked workers ignore use_openmp_fit and always use the
# single-threaded C builder, specifically BECAUSE a forked child
# starting its own OpenMP region after this same process has already
# run OpenMP (subtests 3/4 above did, via use_openmp_fit) can hang.
# This subtest runs deliberately late in the file, after use_openmp_fit
# has already executed in-process above, to exercise exactly that
# ordering.
# ------------------------------------------------------------------------
subtest 'use_openmp_fit + parallel_fit does not hang and is reproducible' => sub {
plan skip_all => 'no fork() on this platform'
unless $can_fork && $HAS_C && $HAS_OPENMP;
( run in 0.618 second using v1.01-cache-2.11-cpan-c966e8aa7e8 )