Acme-Parataxis

 view release on metacpan or  search on metacpan

lib/Acme/Parataxis/Stream.pod  view on Meta::CPAN

=pod

=encoding utf8

=head1 NAME

Acme::Parataxis::Stream - Chainable Async FRP Streams over Bounded Channels

=head1 SYNOPSIS

    use v5.40;
    use Acme::Parataxis qw[async];
    use Acme::Parataxis::Channel;
    use Acme::Parataxis::Stream;

    async {
        my $raw = Acme::Parataxis::Channel->new( capacity => 1024 );

        # Build and consume an asynchronous stream pipeline:
        my $sink = Acme::Parataxis::Stream->from_channel($raw)
            ->map(     sub ($json)  { decode_json($json)      } )
            ->filter(  sub ($item)  { $item->{priority} > 1   } )
            ->throttle(100)                                   # max 100 items/sec
            ->batch_time(1000)                                # group into 1-sec batches
            ->consume( sub (@batch) { db_bulk_insert(@batch) } );

        # Produce data into the source channel:
        $raw->put($_) for @events;
        $raw->shutdown;

        # Await the pipeline sink until all stages drain:
        $sink->await;
    };

=head1 DESCRIPTION

An C<Acme::Parataxis::Stream> is a chainable pipeline of asynchronous processing stages backed by bounded
L<Acme::Parataxis::Channel>s.

Each stage in a stream pipeline is a factory: calling C<map>, C<filter>, C<batch>, C<batch_time>, or C<throttle>
allocates a private bounded output channel, spawns an isolated background fiber to process items, and returns a new
C<Stream> object wrapping that output channel. Calling C<consume> terminates the pipeline and begins draining items.

Key architectural guarantees:

=over

=item *

B<Automatic backpressure>: Output channels between stages are strictly bounded (governed by C<stage_capacity>). If a
stage processes items slower than its upstream feeder, the intermediate channel fills and the upstream fiber parks.
Backpressure propagates naturally all the way to the raw data source without unbounded memory consumption.

=item *

B<Cascading graceful teardown>: Pipelines shut down cleanly from head to tail. When the source channel shuts down, the
first stage reads C<undef> from its input, flushes pending work, and shuts down its output channel. This cascade
propagates downstream, unwinding all intermediate fibers until C<consume> finishes.

=item *

B<Cancellable stages>: Because stages park on standard channels, enclosing C<with_timeout> scopes and cancellation
tokens interrupt in-flight stages cleanly.



( run in 0.761 second using v1.01-cache-2.11-cpan-234fa220139 )