view release on metacpan or  search on metacpan

README  view on Meta::CPAN

    shipped and served read-only. It takes the write lock (so no mutation is
    in flight), sets a seal flag in the header, and flushes it to disk
    ("msync"). Afterwards every mutator on that handle croaks and the handle
    itself becomes read-only. A sharded map seals every shard file. Freezing
    is one-way; there is no unfreeze.

    "new_readonly" opens an already-frozen file with "O_RDONLY" and maps it
    "PROT_READ". Because a sealed map is immutable and can have no writers,
    queries take no lock at all -- no reader-slot bookkeeping, no LRU clock
    bit, no lazy TTL cleanup -- and never write the mapping. A read-only view
    therefore works from a read-only file or filesystem, and any number of
    processes can share one frozen file at once. All queries and full
    iteration are supported: "get", "exists", "get_with_ttl", "get_multi",
    "keys", "values", "items", "to_hash", "each", and cursors ("cursor" and
    "cursor_next"). Every mutator croaks (including the integer counters
    "incr"/"decr"/"max"/"min", which on a writable map update in place under
    the read lock). "sync" is a silent no-op. "frozen" and "readonly" report
    the state, and "stats" gains matching "frozen" and "readonly" keys.

    The seal reuses a previously reserved header byte, so the on-disk format
    and version are unchanged: a file written by an older release is simply
    not frozen (the byte reads 0) and opens read-write exactly as before.

    The two modes never mix: opening a frozen file read-write ("new",
    "new_from_fd") is refused -- open it with "new_readonly" instead -- and
    "new_readonly" refuses a file that has not been frozen (its lock-free
    readers must never race a live writer). "new_readonly" is for a single
    backing file, not a sharded set.

    Portability: a frozen file is a raw memory image. Read it back on the same
    architecture that wrote it (same word size and endianness; the native
    magic and variant id reject a mismatched or wrong-variant file at attach
    time). Ship it by copying the file; do not serve it over NFS or another
    network filesystem while another host has it mapped.

  Crash Safety
    If a process dies (e.g., SIGKILL, OOM kill) while holding the write lock,
    other processes detect the stale lock within 2 seconds and automatically
    recover. The writer's PID is encoded in the rwlock word itself (single
    atomic CAS, no crash window). On "FUTEX_WAIT" timeout, waiters "kill($pid,
    0)" the holder and CAS-release the lock if it's dead.

    Reader-side recovery uses a 1024-slot table in the shared mmap (one slot
    per process, claimed lazily on first lock; fork()'d children claim a fresh
    slot via "pthread_atfork"). A reader's entire contribution to the lock is
    the "rdepth" word in its own slot -- there is no shared reader counter --
    so a dead reader is neutralised by clearing that one slot's PID, and a
    draining writer does so unconditionally as it scans. Because no orphaned
    count can exist, there is no quiescent force-reset: a worker killed
    mid-"incr_by" cannot pin the lock at all. An occupancy bitmap (one bit per
    slot, published before the slot can hold a lock) lets the writer visit
    only occupied slots rather than all 1024. Beyond 1024 simultaneous handles
    per map, a handle that cannot claim a slot proceeds "slotless"; see
    "Reader-slot exhaustion" for the one case that recovery cannot cover.

    The same path validates and rebuilds the LRU doubly-linked list if a dead
    writer left it inconsistent. "stat_recoveries" in "stats" counts every
    recovery event.

    Recovery uses "kill($pid, 0)" for liveness, which cannot distinguish a
    reused PID from the original. Hitting a false "alive" requires a process
    to die in the brief window it holds a read lock and the kernel to cycle
    through the entire PID space back to that exact number within the
    ~2-second recovery window and hand it to a long-lived process -- i.e. a
    runaway fork storm. Even then the effect is bounded: writers stall until
    the recycled process exits; reads are unaffected and no data is corrupted.
    Writer-crash recovery is immune (the writer PID lives in the lock word and
    is reclaimed independently of the slot table).

    Limitation: PID-based recovery assumes all processes share the same PID
    namespace. Cross-container sharing (different PID namespaces) is not
    supported.

    After recovery from a mid-mutation crash, the map data may be partially
    inconsistent (e.g., one entry was being updated when the writer died). Map
    structure (locks, LRU, free lists, counters) is restored, but the specific
    entry being mutated may have stale or partial bytes. Calling "clear" after
    detecting a stale lock recovery is recommended for safety-critical
    applications.

  Reader-slot exhaustion
    Dead-process recovery attributes a crashed lock holder's contribution
    through its reader-slot. The slot table holds 1024 entries (one per
    concurrent reader process). If more than that many reader processes share
    one mapping at once, a reader that cannot claim a slot proceeds "slotless"
    -- it still takes the read lock but leaves no per-process record. If such
    a slotless reader is then killed while holding the read lock, its share of
    the lock cannot be attributed to a dead process, so writer recovery cannot
    reclaim it and writers may block until the mapping is recreated. Reaching
    this needs more than 1024 concurrent reader processes on one mapping plus
    a crash in the brief read-lock window; the dead-process slot reclaim keeps
    the table from filling with stale entries, so in practice it is very
    unlikely.

BENCHMARKS
    Throughput versus other shared-memory / on-disk solutions, 25K entries,
    single process, Linux x86_64. All values in M ops/s (higher is better).
    Run "perl -Mblib bench/vs.pl 25000" to reproduce.

    Integer key - integer value> (Shared::II):

                  BerkeleyDB   LMDB   Shared::II
        INSERT          31       46         184
        LOOKUP          35       40         383
        INCREMENT       16       18         165

    String key - string value, short> (inline <= 7B, Shared::SS):

                  FastMmap   BerkeleyDB   LMDB   SharedMem   Shared::SS
        INSERT        11          26       40        62          130
        LOOKUP        10          32       34       146          213
        DELETE        14          18       --        32           68

    String key - string value, long> (~50-100B, Shared::SS):

                  BerkeleyDB   LMDB   SharedMem   Shared::SS
        INSERT        25         37        61          133
        LOOKUP        30         33       125          229

    LRU cache lookup (25K entries, lock-free clock eviction):