AmberDB

 view release on metacpan or  search on metacpan

Changes  view on Meta::CPAN

          * Direct 64-bit Uint Indexing: Non-foreign key numeric fields in .fld bypass synthetic dictionary ID generation, indexing pure 64-bit unsigned integers directly into binary keys.
          * Batch Foreign Key Pre-Fetching: Integrated batch RDBM ID pre-fetching in search and junk indexing pipelines, reducing disk I/O overhead to zero during keyword tokenization.

        - [LEGACY TABLE MIGRATION & RECONSTRUCTION ENGINE] Automated update_table & update_all in AmberDB::Tools:
          * Multi-Era Format Detection: Implemented _detect_record_format and decode_legacy_record recognizing and decoding all historical formats across AmberDB history: 2003 FlatDB (v1), 2004-2006 \T arrays (v2), 2019-2025 <TAB0>..<TAB3> hierarchic...
          * Automated Timestamped Backups: Automatically backs up migrating tables as <table_name>-v<detected_ver>-<YYYY-MMDD>.db prior to rewriting.
          * Authoritative Data Preservation:
            - .unq (Unique & Synonym Dictionary): Identified as non-reconstructible authoritative master data; strictly exempted from derived index cleanup, preserved live, and snapshot-copied to <table_name>-v<ver>-<date>.unq.
            - .del (Soft-Deleted Records Archive): Detected via exist_table($table, 'del'), backed up, and all archived records migrated to ABR v5.
            - .aut (Audit Trail Log): Detected via exist_table($table, 'aut'), backed up, and user modification history migrated to ABR v5.
            - .cnt (Read Counter): Detected via exist_table($table, 'cnt'), backed up, and live counter state preserved.
          * Clean Re-indexing via insert_list: Cleans up all derived and legacy indexes (qw(inx src fld fac slg srt jinx jfld jsrc)) and rebuilds table data and indexes atomically through $adb->insert_list.

        - [COMMAND LINE UTILITY] Automated Migration Script (bin/update_tables.pl):
          * Added CLI migration script bin/update_tables.pl supporting --all, --table=<names>, --dbase=<dir>, --force, and --help options.
          * Provides granular per-table progress reporting and comprehensive post-migration summary (tables processed, upgraded, already up-to-date, record counts, and companion file backups).

        - [TRANSACTION SAFETY, SEARCH & CSV COMPATIBILITY]
          * Transact Journal Escaping: Escaped \n, \r, \x1e, and \\ in binary record payloads logged to .txn journals, preventing binary length bytes from splitting WAL records or corrupting field boundaries.
          * CSV Line Preservation: In tie2csv and vacuum, exported CSV records using _db_encode_legacy so exported files remain clean single-line human-readable text.
          * Unindexed Search Word Extraction: Fixed search_table in unindexed mode to decode record fields before passing them to get_words, preventing binary length bytes from corrupting search tokens.

        - [QUERY ENGINE & INDEX PERFORMANCE OPTIMIZATIONS] Adaptive Binary Intersect & Pure-Index Candidate Filtering:
          * Adaptive Binary Search Pruning (bin_crop): Introduced dynamic thresholding between XS unpack probing and 8-byte aligned O(log N) binary search (substr). For massive posting lists (e.g. 600K records), eliminates huge Perl scalar allocation...
          * Pure-Index Candidate Probing (search_table): Refactored multi-value/range filtering (e.g. 27-year date intervals) to evaluate candidate IDs directly against sorted 8-byte aligned index buffers via binary search, completely eliminating slo...
          * Batch Unindexed Fallback: Replaced iterative table_readid loops in unindexed search fallbacks with single-pass read_list batching, eliminating repetitive open/close file descriptor syscalls.

        - [TEST COVERAGE & PACKAGING]
          * Added comprehensive test suite t/amberdb_update_table.t covering multi-era format decoding, update_table migration, backup naming, .del/.aut/.cnt/.unq handling, and update_all batch discovery.
          * Updated MANIFEST to include bin/update_tables.pl and new test suites.
          * All 47 test files (440 assertions) passing with 100% success rate.

5.23.2  2026-09-03
        - [LOCALE ENGINE & MULTILINGUAL ARCHITECTURE] 10th Language - Global Base (gb) and Default Locale:
          * Introduced Global Base (gb) as the 10th supported language and new universal default/fallback locale (replacing en).
          * Implemented comprehensive multilingual Latin character preservation in alphabet_chars across European, Turkish, Nordic, French, German, Spanish, and Slavic-Latin alphabets.
          * Added cross-lingual, accent-tolerant search regex mapping (regex_map) matching accented and unaccented variations (e.g. cafe matches café, munchen matches münchen, seker matches şeker).
          * Implemented canonical accent folding in accent_map for high-recall inverted search indexing (.src).
          * Added lossless Unicode ligature conversion in ascii_map (ß->ss, æ->ae, œ->oe, ı->i, ø->o, ł->l, đ->d, ð->d, þ->th, ə->e) for clean URL slug and ASCII ID generation.
          * Configured international English numbering, date formatting, and ISO standard decimal/group separators.
          * Added language aliases: 'gb', 'global', 'gl', 'universal', 'uni', 'gb_base'.
          * Added dedicated test coverage in t/amberdb-locale_09_gb.t.
        - [TRANSACTION ARCHITECTURE & API REFINEMENT] Pure File-Path Error Model and Operational Rollback:
          * Refactored transact_error($file_path, $message) to exclusively accept physical file paths; eliminated artificial "transaction" and "system" string contexts.
          * Simplified table identification via single exact regex /([^\/\\:]+)\.$db_ext$/: directly extracts table ID and inspects schema no_transact attribute; non-db extensions (.inx, .src, .fld, .fac, .slg, .aut, .del, .txn) never trigger rollbac...
          * Added immediate early return in transact_error if $file_path is undefined or empty.
          * Established strict API role separation: application code directly invokes transact_rollback() for business logic cancellations (insufficient stock, credit limits, validation aborts) and unexpected eval exceptions; transact_error is reserv...
          * Unified legacy is_index and is_no_transact flags into single no_rollback attribute.
          * Restructured insert_id to strictly validate mandatory $tableid at entry prior to resolving table paths, with proper ref guard for $rid.

5.23.1  2026-09-02
        - [DISTRIBUTION & METADATA] Unified Versioning and MetaCPAN Curation:
          * Unified $VERSION across all internal and standalone modules to 5.23.1 for consistent release identification and stack-trace debugging.
          * Added no_index -> file specification in Makefile.PL for internal engine modules (Base, Cache, Index, Transact, Locale::Currency), curating the MetaCPAN release page to highlight the 6 public standalone modules.
          * Excluded wiki/ documentation directory from CPAN distribution package (via MANIFEST.SKIP), reducing tarball size by ~30% and eliminating 246 redundant packaging entries.
          * Updated main AmberDB abstract to "High-performance embedded NoSQL database engine for Perl".

5.23.0  2026-09-02
        - [ARCHITECTURE & ID SIMPLIFICATION] Pure 64-bit Binary Engine & Deprecation of 8-Byte ASCII IDs:
          * Deprecated 8-byte ASCII IDs (a8) across binary index structures in favor of pure 64-bit Big-Endian unsigned integer packing (Q>*), guaranteeing O(1) binary slicing with fixed 8-byte record strides.
          * Simplified bin_encode() and bin_decode() by removing fragile \0 byte auto-detection and string unpacking heuristics.
          * Removed legacy id_type schema attribute across all modules and test suites; standard relational tables strictly enforce positive integer numeric IDs.
        - [STORAGE & PER-TABLE SIMPLE MODE] Hybrid Multi-Model Architecture with use_simple => 1:
          * Introduced per-table use_simple => 1 attribute allowing key-value tables with arbitrary string keys up to 255 bytes (UUIDs, session tokens, emails, slugs, etc.) alongside standard relational tables.
          * Preserved canonical physical table path ($dbase_dir/table/$table.db) for use_simple tables in standard database mode.
          * Selective schema sanitization: tables with use_simple => 1 strip indexing, columnar, and caching definitions (blocks, match_block, search_block, facet_block, sort_block, record_index, use_cache, cache_ttl) to achieve zero index/cache I/O ...
          * Enabled keep_deleted => 1 archiving (.del) on use_simple tables.
        - [RDBM & INTEGRITY] Foreign Key and Cross-Table Isolation:
          * Prohibited standard relational tables from binding foreign keys (RDBM) to use_simple tables in rdbm_target() and _resolve_field_value().
        - [TESTS & VERIFICATION] Dedicated Test Coverage:
          * Added comprehensive test suite t/amberdb_table_use_simple.t verifying arbitrary string keys, schema sanitization, zero index file creation, keep_deleted archiving, and RDBM isolation.
          * Cleaned up legacy id_type occurrences across all 41 test files and concurrency stress tests (xt/amberdb_concurrency_stress.t).
        - [DISTRIBUTION & METADATA] Decoupled Internal Module Versions and CPAN no_index:
          * Stripped redundant $VERSION definitions from 15 internal/engine-only modules (Base, Cache, Transact, Index, Index::Facet, Index::Junk, Locale::Currency, Locale::Lang::*) to streamline release maintenance.
          * Retained explicit $VERSION in public, standalone modules (AmberDB, AmberDB::Array, AmberDB::Date, AmberDB::Locale, Amber::Util::String, AmberDB::Tools).
          * Added comprehensive META_MERGE no_index configuration in Makefile.PL covering internal directories, namespaces, and packages.
        - [DATA STRUCTURES & UTILITIES] Non-Destructive hash_diff in AmberDB::Array:
          * Implemented hash_diff($hash1, $hash2) in AmberDB::Array with safe shallow copying, ensuring input hash structures are never mutated.
          * Removed legacy internal _hash_diff from AmberDB::Tools and switched to $adb->hash_diff directly.
          * Added unit test coverage for hash_diff in t/amberdb_array.t.

5.22.2  2026-09-01
        - [CROSS-PLATFORM & CI] Universal GitHub Actions CI Matrix:
          * Configured multi-platform CI matrix testing on Linux (ubuntu-latest), macOS (macos-latest), and Windows (windows-latest) across Perl 5.16 through 5.40.
          * Configured official Strawberry Perl distribution in CI for native Berkeley DB / DB_File binary compatibility and prevented Git Bash PATH collisions.
        - [STORAGE & TRANSACTIONS] Safe Directory Scanner and Windows File Sharing:
          * Introduced dir_files($dir, [$pattern], [%opts]) helper method in AmberDB::Base for cross-platform, safe file discovery supporting wildcards (*.db, txn_*.txn) and compiled regular expressions (qr/../).
          * Refactored AmberDB::Transact and AmberDB::Tools (dir_tables, all_tables, del_table) to use dir_files, completely removing fragile glob usage.
          * Fixed Windows NTFS file sharing collision in transact_recover: journal lines are now read directly from open locked handles, preventing secondary open permission denied errors.
        - [TESTS & STABILITY] Test Concurrency and Monotonic Sequence:
          * Hardened t/amberdb_transact.t orphan recovery subtest to use strictly monotonic table_autoid record IDs (65) and authentic transaction lifecycle simulation.
          * Guaranteed complete handle and lock cleanup with $adb->close_all() in crash recovery test scenarios.

5.22.1  2026-09-01
        - [DOCS] Documentation and Synopsis Fixes:
          * Fixed synopsis and API example signatures across documentation and POD.
          * Updated README.md installation instructions with streamlined CPAN/cpanm cross-platform support.
          * Fixed character encoding and wide-character warnings in test suite schemas.

5.22.0  2026-08-31
        - [REFACTOR] Codebase Cleanliness:
          * Removed redundant single-line helper is_rdbm_block() in favor of direct rdbm_target() calls across AmberDB::Index, AmberDB::Index::Junk, and AmberDB::Tools.
        - [RDBM & INTEGRITY] Hardened Foreign Table Auto-Registration in field_to_list():
          * Replaced decoupled table_autoid() and recs_put() sequence with atomic insert_id($target_table, 0, @record) to prevent ID race conditions.
          * Corrected foreign record column alignment so auto-registered values are accurately placed at block index $target_blk rather than hardcoded column 1.
          * Guaranteed that foreign table primary indexes (.inx) and secondary indexes (.unq, .src, .fac, .slg) are fully and consistently constructed upon auto-registration.
        - [SECURITY & VALIDATION] Universal ID Verification and Simple Mode Key Sanitization:
          * Eliminated opt-in config('id_check') requirement: every schema table now strictly enforces its schema id_type (positive integers for num, safe 8-byte chars for ascii).
          * Enforced scalar ID requirement: any reference (ARRAY ref, HASH ref, etc.) passed as a record ID is strictly rejected across all modes.
          * Introduced Safe Key Sanitization in Simple Mode: automatically trims leading/trailing whitespace, strictly rejects NUL bytes (\0) and control characters (\r, \n, \t, \x00-\x1F, \x7F) that corrupt Berkeley DB or CSV backups, and enforces a...
          * Integrated unconditional id_check across table_autoid(), read_id(), modify_id(), delete_id(), and exist_id().
        - [REFACTOR] Terminology and Codebase Refactoring: SEO -> Slug:
          * Completely replaced SEO terminology with Slug across the entire codebase, test suite, and documentation without legacy aliases:
            - Renamed methods: set_seourl() -> set_slug(), get_seourl() -> get_slug().
            - Renamed schema attributes: seo_block -> slug_block, seo_max_len -> slug_max_len.
          * Renamed test suite t/amberdb_seo_facet_bulk.t to t/amberdb_slug_facet_bulk.t.
        - [STORAGE] Standardized URL Slug Map File Extension: .rwt -> .slg:
          * Renamed binary slug map files from _0.rwt / _1.rwt to _0.slg (ID -> Slug) and _1.slg (Slug -> ID) across AmberDB, Index, Transact, and Tools.
          * Updated Tools->set_index() to reconstruct .slg files with zero data loss.
        - [DOCS] Caching Terminology Unification:
          * Completely removed deprecated L1/L2 caching terminology from all documentation and POD.
          * Standardized on unified "Shared RAM-Disk (tmpfs / ImDisk) Cache" architecture.
        - [TESTS & CONCURRENCY] Cross-Platform Multi-Process Concurrency Stress Testing:
          * Upgraded xt/amberdb_concurrency_stress.t to an OS-aware unified architecture supporting both Linux (POSIX fork+exec) and Windows (independent asynchronous perl.exe processes).
          * Ensures full operating system-level flock(LOCK_EX) lock isolation on Windows without CRT/pseudo-fork thread contention.
          * Comprehensive validation across 5 concurrency scenarios:
            1. Multi-worker parallel writes with secondary index synchronization (.inx, .fld, .src, .fac, .srt, .slg).
            2. Interleaved concurrent reads and writes without deadlocks or corruption.
            3. Concurrent independent transactions with simulated mid-operation process crashes and orphan recovery (transact_recover).
            4. High-concurrency duplicate title insertion verifying deterministic URL slug collision resolution and 100% bidirectional bijection.
            5. Shared record inventory decrements using record-level locks (flock_open / flock_close).
        - Added Section 7.8 ("Multi-Process Concurrency, Lock Isolation, and Stress Verification") to Turkish and English documentation.
        - Updated README.md with concurrency stress testing instructions.

5.21.1  2026-08-29
        - Fixed read_id return signature across documentation examples to consistently return array format.
        - Streamlined transaction workflow documentation to use transact_error() for business validations with automatic commit/rollback in transact_end().
        - Clarified field_fetch return signatures in English documentation to reflect full record retrieval.
        - Declared missing core dependencies (Archive::Tar, Digest::SHA, JSON::PP, Hash::Util) in Makefile.PL and cpanfile.

5.21.0  2026-08-28
        - Fixed schema cache poisoning in restore() and hardened dbase_info()/table_info() against caching empty parse results.
        - Standardized instance variable naming across codebase, test suites, and documentation to $adb (AmberDB Handle):
          * Introduced $adb->config() method supporting scalar get, defensive copy bulk get, and side-effect hook execution (locale reloading, path cache invalidation).
          * Introduced $adb->path() method for standardized path retrieval and modification across core modules and scripts.
          * Enhanced $adb->table_attr() with unified getter/setter and automatic path refresh on schema changes (year, section, lang).
          * Protected $adb->table_info() by returning shallow copies to prevent external reference leaking and unauthorized in-memory state mutations.
          * Refactored all internal core modules (lib/AmberDB.pm, lib/AmberDB/Base.pm, lib/AmberDB/Tools.pm, lib/AmberDB/Transact.pm, lib/AmberDB/Cache.pm, lib/AmberDB/Index/Junk.pm) to interact strictly through accessor methods ($adb->config, $adb->...



( run in 2.371 seconds using v1.01-cache-2.11-cpan-b16cb0d3907 )