view release on metacpan or  search on metacpan

README  view on Meta::CPAN

        # String queue (mutex-protected, circular arena)
        my $sq = Data::Queue::Shared::Str->new('/tmp/strq.shm', 1024);
        $sq->push("hello world");
        my $msg = $sq->pop;

        # With explicit arena size (default: capacity * 256)
        my $sq = Data::Queue::Shared::Str->new('/tmp/strq.shm', 1024, 1048576);

        # Multiprocess
        if (fork() == 0) {
            my $child = Data::Queue::Shared::Int->new('/tmp/myq.shm', 1024);
            $child->push(99);
            exit;
        }
        wait;
        print $q->pop;  # 99

DESCRIPTION
    Data::Queue::Shared provides bounded MPMC (multi-producer,
    multi-consumer) queues stored in file-backed shared memory
    (mmap(MAP_SHARED)), enabling efficient multiprocess data sharing on
    Linux.

    Linux-only. Requires 64-bit Perl.

  Variants
    Data::Queue::Shared::Int - int64 values, lock-free (16 bytes/slot)
        Uses the Vyukov bounded MPMC algorithm. Push and pop are lock-free
        (CAS-based). Optimal for integer job IDs, counters, indices.

    Data::Queue::Shared::Int32 - int32 values, lock-free (8 bytes/slot)
    Data::Queue::Shared::Int16 - int16 values, lock-free (8 bytes/slot)
        Compact variants with 32-bit Vyukov sequence numbers. Half the
        memory footprint per slot = double the cache density. Same lock-free
        algorithm. Same API as Int. Values outside the type range are
        silently truncated (standard C cast semantics).

    Data::Queue::Shared::Str - byte string values, mutex-protected
        Uses a futex-based mutex with a circular arena for variable-length
        string storage. Supports UTF-8 flag preservation. Optimal for
        messages, serialized data, filenames.

        Memory-efficient for mixed lengths (short + occasional long strings
        share the arena), but throughput degrades under heavy multi-producer
        contention because all pushes serialize on one mutex.

        For fixed-length FIFO workloads that need maximum throughput: use
        Data::Deque::Shared::Str with only "push_back" / "pop_front". It's
        fixed-slot-per-entry (memory use = capacity × max_len), uses a
        per-slot publication state machine instead of a shared mutex, and
        measures 1.3x-4.7x faster than Queue::Str depending on contention (1
        vs 8 writers, 32-byte payloads, single box). Use that if your
        messages share an upper bound and you want lock-free-style scaling.

  Features
    *   File-backed mmap for cross-process sharing

    *   Lock-free MPMC for integer queues (Vyukov algorithm)

    *   Futex-based blocking wait with timeout (no busy-spin)

    *   PID-based stale lock recovery (dead process detection)

    *   Batch push/pop operations

    *   Circular arena for zero-fragmentation string storage

    *   Optional keyword API via XS::Parse::Keyword (zero method-dispatch
        overhead)

  Constructor
        # Int queue
        my $q = Data::Queue::Shared::Int->new($path, $capacity);

        # Str queue
        my $q = Data::Queue::Shared::Str->new($path, $capacity);
        my $q = Data::Queue::Shared::Str->new($path, $capacity, $arena_bytes);

    Creates or opens a shared queue backed by file $path. $capacity is
    rounded up to the next power of 2. When opening an existing file,
    parameters are read from the stored header. Multiple processes can open
    the same file simultaneously.

    Pass "undef" for $path to create an anonymous queue using
    "MAP_SHARED|MAP_ANONYMOUS". Anonymous queues are shared with child
    processes via fork() but cannot be opened by unrelated processes.

  memfd Constructor
        my $q = Data::Queue::Shared::Int->new_memfd($name, $capacity);
        my $q = Data::Queue::Shared::Str->new_memfd($name, $capacity);
        my $q = Data::Queue::Shared::Str->new_memfd($name, $cap, $arena);

    Creates a queue backed by memfd_create(2). No filesystem path — the
    backing memory is identified by a file descriptor. Use memfd() to
    retrieve the fd and pass it to other processes via "SCM_RIGHTS" (Unix
    domain socket fd passing) or fork() inheritance.

        my $q2 = Data::Queue::Shared::Int->new_from_fd($fd);
        my $q2 = Data::Queue::Shared::Str->new_from_fd($fd);

    Opens a queue from a received memfd. The fd is dup'd internally.

        my $fd = $q->memfd;    # backing fd (-1 if file-backed/anonymous)

    For Str queues, $arena_bytes sets the string storage arena size
    (default: "$capacity * 256", minimum 4096, maximum 4GB). Strings are
    stored in a circular arena; total stored string bytes cannot exceed the
    arena capacity. Individual strings are limited to ~2GB.

  API
   Core operations
        my $ok  = $q->push($value);             # non-blocking, false if full
        my $val = $q->pop;                       # non-blocking, undef if empty
        my $ok  = $q->push_wait($value);         # blocking, infinite wait
        my $ok  = $q->push_wait($value, $secs);  # blocking with timeout
        my $val = $q->pop_wait;                  # blocking, infinite wait
        my $val = $q->pop_wait($secs);           # blocking with timeout
        my $val = $q->peek;                      # read front without consuming

    "peek" returns the front element without removing it ("undef" if empty).