Acme-Parataxis
view release on metacpan or search on metacpan
lib/Acme/Parataxis/Stream.pod view on Meta::CPAN
=pod
=encoding utf8
=head1 NAME
Acme::Parataxis::Stream - Chainable Async FRP Streams over Bounded Channels
=head1 SYNOPSIS
use v5.40;
use Acme::Parataxis qw[async];
use Acme::Parataxis::Channel;
use Acme::Parataxis::Stream;
async {
my $raw = Acme::Parataxis::Channel->new( capacity => 1024 );
# Build and consume an asynchronous stream pipeline:
my $sink = Acme::Parataxis::Stream->from_channel($raw)
->map( sub ($json) { decode_json($json) } )
->filter( sub ($item) { $item->{priority} > 1 } )
->throttle(100) # max 100 items/sec
->batch_time(1000) # group into 1-sec batches
->consume( sub (@batch) { db_bulk_insert(@batch) } );
# Produce data into the source channel:
$raw->put($_) for @events;
$raw->shutdown;
# Await the pipeline sink until all stages drain:
$sink->await;
};
=head1 DESCRIPTION
An C<Acme::Parataxis::Stream> is a chainable pipeline of asynchronous processing stages backed by bounded
L<Acme::Parataxis::Channel>s.
Each stage in a stream pipeline is a factory: calling C<map>, C<filter>, C<batch>, C<batch_time>, or C<throttle>
allocates a private bounded output channel, spawns an isolated background fiber to process items, and returns a new
C<Stream> object wrapping that output channel. Calling C<consume> terminates the pipeline and begins draining items.
Key architectural guarantees:
=over
=item *
B<Automatic backpressure>: Output channels between stages are strictly bounded (governed by C<stage_capacity>). If a
stage processes items slower than its upstream feeder, the intermediate channel fills and the upstream fiber parks.
Backpressure propagates naturally all the way to the raw data source without unbounded memory consumption.
=item *
B<Cascading graceful teardown>: Pipelines shut down cleanly from head to tail. When the source channel shuts down, the
first stage reads C<undef> from its input, flushes pending work, and shuts down its output channel. This cascade
propagates downstream, unwinding all intermediate fibers until C<consume> finishes.
=item *
B<Cancellable stages>: Because stages park on standard channels, enclosing C<with_timeout> scopes and cancellation
tokens interrupt in-flight stages cleanly.
( run in 0.761 second using v1.01-cache-2.11-cpan-234fa220139 )