AmberDB
view release on metacpan or search on metacpan
* Fixed test emulation fallback in AmberDB::Base::Ramdisk when database name is omitted under $ENV{AMBERDB_TEST_RAMDISK}, defaulting to isolated tmpdir()/amberdb_test to ensure clean schema inheritance testing (t/amberdb_dbase_inheritance.t...
* Updated AmberDB::Tools::Update::update_storage to exclude the obsolete 'ramdisk' folder from canonical directory layouts.
* Purged all orphaned physical dbstore/ramdisk directories across active projects.
* Standardized lock directory resolution in t/amberdb_lock.t using $adb->path('lock_dir').
* Added comprehensive R1 dual-write synchronization assertions in xt/amberdb_ramdisk.t verifying .inx, .fld, and .slg physical persistence.
- [CONNECT PARAMETER NORMALIZATION]
* Supported 'db' parameter alias in AmberDB constructor alongside database, dbase, and dbname.
- [DYNAMIC FIELD NORMALIZATION ENGINE (.norm)]
* Implemented field_normalize() and norm_info() in AmberDB::Base::Schema for domain-agnostic feature extraction from unstructured raw text strings.
* Normalization rules reside in dedicated <table_id>.norm schema files, eliminating runtime overhead for ordinary read/query workloads.
* Supports pipeline stages: preprocess boundary regexes, categorical mapping, single matches (integer, titlecase, trim, code callbacks), and multi-matches (min/max selection).
* Optional schema block array mapping via { as_array => 1 }.
* Registered _norm container in AmberDB core constructor with lock_keys and lock_value protection.
* Added comprehensive unit test suite in t/amberdb_field_normalize.t.
- [INDEX INTEGRITY & STRICT NUMERIC ID REINDEXING]
* Enforced strict positive integer filtering (/^\d+$/ && $_ > 0) across all binary index buffers (keys, A:keys, B:keys, $blk:keys, bin_encode, bin_decode, bin_add, bin_punch), completely preventing non-numeric or non-positive IDs from enter...
* Updated set_index in AmberDB::Tools::Index to close and unlink all existing index files (.inx, .src, .fld, .fac, .slg, .unq) on both physical disk and RAM-disk before rebuilding, ensuring clean and deterministic recreation.
* Added RAM-disk synchronization to set_index: freshly rebuilt index files are atomically mirrored to RAM-disk upon completion with safe filehandle management.
* Fixed UTF-8 double-encoding in $blk:vals across sort_add, sort_modify, and sort_del in AmberDB::Base::Index to prevent exponential byte bloat on multibyte character updates.
* Enhanced amberdb_cli.pl with database name auto-detection from connect.pl, core.conf (site_id), and project directories, ensuring CLI maintenance commands attach to active RAM-disk instances.
* Disallowed generic 'dbstore' as a valid database name across Ramdisk and CLI modules to eliminate unintended directory creation on RAM-disk.
* Enforced strict CLI connect argument validation: positional argument after 'connect' is strictly the database name (amberdb connect <database_name> [user] [pass]). Explicitly rejected 'connect dbstore' and directory paths with a descripti...
* Added non-numeric key and record ID enforcement in index_put(): when $type is not 'raw', non-numeric IDs in array references are filtered out and non-numeric scalar values are rejected from entering index files (.inx, .src, .fld, etc.).
- [FAST RANGE FILTER ENGINE]
* Implemented range_slice() using pre-sorted .inx keys ($blk:keys) and .fld boundaries for sub-millisecond range queries.
* Slices contiguous 8-byte packed binary record IDs directly via index() and substr(), replacing slow multi-key OR unions.
* Stored distinct block values under $blk:vals (and tiered A:$blk:vals, B:$blk:vals) in .inx for binary search boundary detection.
* Added candidate pruning for small survivor sets (<= 250 records) directly on the raw binary slice.
* Full lifecycle management for $blk:vals: automatically maintains and prunes values across sort_add, sort_modify, and sort_del.
- [FIELD QUERY ENHANCEMENTS]
* field_fetch now accepts named block arguments via resolve_block_idx().
* Fixed raw numeric value recovery for fallback queries on normalized .inx sort keys.
- [DATE ENGINE OPTIMIZATIONS]
* Added memoization caching to AmberDB::Date with rollover protection.
* Added reset_date() methods.
* Replaced direct hash access with fast accessor methods across dump, vacuum, and backup routines.
- [DIRECTORY ARCHITECTURE & LOCK PATH STANDARDIZATION]
* Standardized lock_dir and session_dir as single-key directories in AmberDB::Base::Ramdisk and AmberDB::Base.
* Removed obsolete fallback keys (lock_rdir) and legacy naming aliases (conf_dir, cfg_dir).
* In RAM-disk environments, lock_dir and session_dir are dynamically mounted under $ramdisk_dir/lock and $ramdisk_dir/session with 0755 permissions; when unmounted, they reside cleanly under $dbase_dir/lock and $dbase_dir/session.
* Aligned database lock directory structures with Misk 4.x state locking specifications.
- [SCHEMA LOADING RESILIENCE]
* Wrapped 'do $target_path' inside eval { do $target_path } in AmberDB::Base::Schema (table_info() and dbase_info()).
* Prevents fatal process/worker termination when user .table or .dbase schema files contain syntax errors under global $SIG{__DIE__} handlers.
- [SLUG INDEX ARCHITECTURE VERIFICATION (.slg)]
* Verified flat unified .slg index access model where block numbers (formerly _0 / _1 file suffixes) reside in key prefixes:
- Block 0: "0:$rid" -> URL slug value
- Block 1: "1:$slug" -> Record ID value
* Enforced direct .slg table access and recs_get key lookups without legacy .rwt fallbacks.
- [GROUP-LEVEL SCHEMA INHERITANCE (.dbase -> .table)]
* Database group schema files (.dbase) now cascade common operational flags to all member tables in the group.
* Added %INHERITABLE_DBASE_KEYS whitelist in AmberDB::Base::Schema (use_ramdisk, keep_deleted, use_counter, log_owner, table_dir, use_section, etc.) preventing table-specific blocks and rules from leaking.
* Granular overrides: attributes explicitly defined in a .table file take precedence over .dbase defaults.
* Upgraded version string to 5.25.3 across all modules in lib/.
5.25.2 2026-09-11
- [CONNECTION PROFILE & SHADOW CRYPTOGRAPHY ARCHITECTURE]
* Decentralized connection profile abstraction ($adb->{_connect}) managing database, username, password, and session token.
* Introduced Unix shadow format standard: sha256$<16_hex_salt>$<hex_digest> via Digest::SHA for one-way password protection.
* Self-securing auto-elevation: plain-text passwords written to connect.pl are automatically upgraded to salted SHA-256 shadow hashes upon connection, and plain-text passwords are purged from disk.
* Full backwards compatibility: empty/undef password and shadow grant passwordless access for local application layers (web, local workers).
* Automated connect.pl bootstrapping per database store under $dbase_dir/config/connect.pl with canonical system profiles (admin, web, cli).
* Session management encapsulation: delegated 4-digit token generation, session persistence, collision avoidance, and cleanup from CLI directly to AmberDB::Base (connect(), disconnect(), generate_token(), session_file()).
* User management API: added user_add(), user_passwd(), user_del(), user_list(), and user_verify() to AmberDB::Base.
* CLI Integration: added amberdb connect /path/to/dir <database> <user> <pass> and user management console (amberdb user list/add/passwd/del).
* Enforced actor username as 'cli' in bin/amberdb_cli.pl: all CLI sessions and commands strictly operate under the 'cli' client profile regardless of incoming username parameters.
- [RAM-DISK ARCHITECTURE & MOUNT MANAGEMENT REFACTORING]
* Centralized candidate RAM-disk directory resolution and OS detection inside ramdisk_is_mounted().
* Standardized platform detection across AmberDB::Base::Ramdisk to canonical 'windows', 'macos', or 'linux'.
* Generalized macOS APFS/HFS+ volume detection to match any mounted volume under /Volumes/ (e.g. /Volumes/AmberDB_RAM, /Volumes/eticaretim, /Volumes/<User>).
* Implemented encapsulated RAM-disk mount status caching with get_cache()/set_cache('ramdisk', 'mount') and support for force => 1 re-verification.
* Cleaned up ramdisk_setup() to cleanly fall back to dbase_dir and physical directories when unmounted, while dynamically repointing and creating table_rdir, schema_rdir, conf_rdir, lock_dir, and session_dir on the RAM-disk when mounted.
* Flexible argument parsing in ramdisk_setup() supporting ($tableid, %opts), (%opts), and hashref options.
* Automatically invoke ramdisk_setup() inside set_datadir() when changing database root directory.
- [CLI SESSION COLLISION AVOIDANCE & CODEBASE CLEANUP]
* Added collision prevention loop in bin/amberdb_cli.pl generate_token() checking existing session files in both project local (.amberdb/session/sess_$token) and active database session_dir (cli_$token).
* Removed redundant if ($adb) checks and legacy candidate search arrays in CLI session handlers.
* Upgraded version string to 5.25.2 across all 25 modules in lib/.
- [TRANSACTION UNDO JOURNAL BASE64 STANDARDIZATION]
* Standardized transaction undo logs (dbstore/journal/txn_*) to strictly encode raw database before-images with Base64 (MIME::Base64).
* Eliminated legacy string regex escaping ($safe_enc/$safe_dec) and index Hex conversions (unpack/pack "H*").
* Unified transaction rollback with RAM-disk Tier 4 (sync_ramdisk) Base64 payload specification for 100% binary safety.
5.25.1 2026-09-11
- [JOURNAL & TRANSACTION STANDARDIZATION] Extensionless Uniform Journaling:
* Standardized all transaction undo logs and RAM-disk write-behind queues under $dbase_dir/journal/ ($adb->journal_dir()).
* Removed legacy .txn extension; transaction journal files now follow the standard extensionless format dbstore/journal/txn_<timestamp>_<seq>_<pid> (Base.pm:481-488).
* Modernized transact_recover() to scan journal/ directory for txn_* files and extract process IDs via /[-_](\d+)$/ for deterministic crash recovery without race conditions.
* Added explicit transact_commit() method alias to transact_end() matching documented public API semantics.
* Cleaned out obsolete legacy 5-parameter signature fallback and 6-field journal format fallback in AmberDB::Base::Transact.
- [RAM-DISK TIER 4 WRITE-BEHIND JOURNALING] Explicit sync_ramdisk Queue:
* Renamed dirty sync queue from sync_events to sync_ramdisk for clear semantic clarity:
- Active Queue: dbstore/journal/sync_ramdisk
- Rotated Queue: dbstore/journal/sync_ramdisk_<timestamp>
* Updated AmberDB::Base::Ramdisk (ramdisk_mark_dirty, ramdisk_sync, ramdisk_sync_db_path) and AmberDB::Base (journal_slot, journal_scan) to reflect sync_ramdisk naming.
* Synchronized background daemon engine (bin/amberdb_daemon.pl) and unit test suites (t/amberdb_ramdisk_tiers.t, t/amberdb_journal.t).
- [MODERN COMMAND LINE INTERFACE & NATURAL QUERY PARSER]
* Implemented unified bin/amberdb_cli.pl with bin/amberdb (POSIX shell) and bin/amberdb.bat (Windows CMD) wrappers.
* Natural positional query signatures:
- amberdb read <table> <id> (read_id)
- amberdb read <table> all [offset] [limit] [modifiers] (read_all)
- amberdb search <table> <query> [offset] [limit] [modifiers] (search_table)
- amberdb fetch <table> <block> <value> (field_fetch)
- amberdb count <table>, amberdb info <table>, amberdb status, amberdb ping
* Dual execution model: direct stateless execution against database without background session side-effects, and isolated session management via 4-digit numeric tokens (connect, disconnect, config, path, attr).
* Trailing output format (json, pretty, tsv, dumper, table, raw) and elapsed execution time (time, time=1, --time) keywords.
* Fixed synopsis and API example signatures across documentation and POD.
* Updated README.md installation instructions with streamlined CPAN/cpanm cross-platform support.
* Fixed character encoding and wide-character warnings in test suite schemas.
5.22.0 2026-08-31
- [REFACTOR] Codebase Cleanliness:
* Removed redundant single-line helper is_rdbm_block() in favor of direct rdbm_target() calls across AmberDB::Index, AmberDB::Index::Junk, and AmberDB::Tools.
- [RDBM & INTEGRITY] Hardened Foreign Table Auto-Registration in field_to_list():
* Replaced decoupled table_autoid() and recs_put() sequence with atomic insert_id($target_table, 0, @record) to prevent ID race conditions.
* Corrected foreign record column alignment so auto-registered values are accurately placed at block index $target_blk rather than hardcoded column 1.
* Guaranteed that foreign table primary indexes (.inx) and secondary indexes (.unq, .src, .fac, .slg) are fully and consistently constructed upon auto-registration.
- [SECURITY & VALIDATION] Universal ID Verification and Simple Mode Key Sanitization:
* Eliminated opt-in config('id_check') requirement: every schema table now strictly enforces its schema id_type (positive integers for num, safe 8-byte chars for ascii).
* Enforced scalar ID requirement: any reference (ARRAY ref, HASH ref, etc.) passed as a record ID is strictly rejected across all modes.
* Introduced Safe Key Sanitization in Simple Mode: automatically trims leading/trailing whitespace, strictly rejects NUL bytes (\0) and control characters (\r, \n, \t, \x00-\x1F, \x7F) that corrupt Berkeley DB or CSV backups, and enforces a...
* Integrated unconditional id_check across table_autoid(), read_id(), modify_id(), delete_id(), and exist_id().
- [REFACTOR] Terminology and Codebase Refactoring: SEO -> Slug:
* Completely replaced SEO terminology with Slug across the entire codebase, test suite, and documentation without legacy aliases:
- Renamed methods: set_seourl() -> set_slug(), get_seourl() -> get_slug().
- Renamed schema attributes: seo_block -> slug_block, seo_max_len -> slug_max_len.
* Renamed test suite t/amberdb_seo_facet_bulk.t to t/amberdb_slug_facet_bulk.t.
- [STORAGE] Standardized URL Slug Map File Extension: .rwt -> .slg:
* Renamed binary slug map files from _0.rwt / _1.rwt to .slg (ID -> Slug) and .slg (Slug -> ID) across AmberDB, Index, Transact, and Tools.
* Updated Tools->set_index() to reconstruct .slg files with zero data loss.
- [DOCS] Caching Terminology Unification:
* Completely removed deprecated L1/L2 caching terminology from all documentation and POD.
* Standardized on unified "Shared RAM-Disk (tmpfs / ImDisk) Cache" architecture.
- [TESTS & CONCURRENCY] Cross-Platform Multi-Process Concurrency Stress Testing:
* Upgraded xt/amberdb_concurrency_stress.t to an OS-aware unified architecture supporting both Linux (POSIX fork+exec) and Windows (independent asynchronous perl.exe processes).
* Ensures full operating system-level flock(LOCK_EX) lock isolation on Windows without CRT/pseudo-fork thread contention.
* Comprehensive validation across 5 concurrency scenarios:
1. Multi-worker parallel writes with secondary index synchronization (.inx, .fld, .src, .fac, .srt, .slg).
2. Interleaved concurrent reads and writes without deadlocks or corruption.
3. Concurrent independent transactions with simulated mid-operation process crashes and orphan recovery (transact_recover).
4. High-concurrency duplicate title insertion verifying deterministic URL slug collision resolution and 100% bidirectional bijection.
5. Shared record inventory decrements using record-level locks (flock_open / flock_close).
- Added Section 7.8 ("Multi-Process Concurrency, Lock Isolation, and Stress Verification") to Turkish and English documentation.
- Updated README.md with concurrency stress testing instructions.
5.21.1 2026-08-29
- Fixed read_id return signature across documentation examples to consistently return array format.
- Streamlined transaction workflow documentation to use transact_error() for business validations with automatic commit/rollback in transact_end().
- Clarified field_fetch return signatures in English documentation to reflect full record retrieval.
- Declared missing core dependencies (Archive::Tar, Digest::SHA, JSON::PP, Hash::Util) in Makefile.PL and cpanfile.
5.21.0 2026-08-28
- Fixed schema cache poisoning in restore() and hardened dbase_info()/table_info() against caching empty parse results.
- Standardized instance variable naming across codebase, test suites, and documentation to $adb (AmberDB Handle):
* Introduced $adb->config() method supporting scalar get, defensive copy bulk get, and side-effect hook execution (locale reloading, path cache invalidation).
* Introduced $adb->path() method for standardized path retrieval and modification across core modules and scripts.
* Enhanced $adb->table_attr() with unified getter/setter and automatic path refresh on schema changes (year, section, lang).
* Protected $adb->table_info() by returning shallow copies to prevent external reference leaking and unauthorized in-memory state mutations.
* Refactored all internal core modules (lib/AmberDB.pm, lib/AmberDB/Base.pm, lib/AmberDB/Tools.pm, lib/AmberDB/Transact.pm, lib/AmberDB/Cache.pm, lib/AmberDB/Index/Junk.pm) to interact strictly through accessor methods ($adb->config, $adb->...
* Enforced restricted hash key access via Hash::Util::lock_keys with private internal naming (_cfg, _path, _table, _dbase, _cache, _db, _txn) and locked container references (Hash::Util::lock_value) against typo/unauthorized overwrites.
* Added comprehensive unit test suite t/amberdb_encapsulation.t covering all 10 encapsulation scenarios.
- [MIGRATION NOTICE / BREAKING CHANGE] Standardized schema terminology across the entire codebase and directory layout:
* UPGRADE ACTION REQUIRED: Existing projects must rename their physical 'dbstore/scheme/' directory to 'dbstore/schema/'.
* Updated RAM-disk setup scripts (setup_ramdisk.sh, setup_ramdisk.ps1, setup_ramdisk.pl, setup_ramdisk.bat) to mount 'schema/'.
- Upgraded transaction engine specification to full ACID-Compliance with Strict Two-Phase Locking (Strict 2PL):
* Enforced Lock-Before-Write and Lock-Before-Read ordering across insert_id, modify_id, and delete_id for true serializable isolation.
* Introduced 'no_transact => 1' schema attribute and table_attr() support to exempt auxiliary tables from abort cascades while preserving LIFO rollback consistency.
- Added comprehensive ACID architectural guarantees section to documentation (README.md, Turkish and English User Guides).
- Clarified architectural distinction between high-throughput batch ETL imports and atomic business transactions.
- Standardized file open error diagnostics and OS-level reporting ($!) across all core modules:
* Added explicit OS error reporting ($!) to all open and tie failures in AmberDB, Base, Cache, Transact, and Tools.
* Replaced silent schema open failure in AmberDB::Base::table_write with diagnostic cluck and graceful return.
* Improved audit log error handling in AmberDB::auth_insert with cluck and record skipping.
- Redesigned 2-Pillar Disaster Recovery and Native Backup Architecture:
* Upgraded recs_back to continuous chronological time-series stream in 'backup/YYYY/YYYY-MM-DD.csv' eliminating folder clutter and ensuring zero-data-loss logging.
* Added Tools->dump() for creating portable, compressed '.amberdb' archives packaging schemas (schema/*.table, schema/*.dbase), authoritative data files (tables/*.db, tables/*.del, tables/*.aut, tables/*.cnt, tables/*_*.str), and cryptograp...
* Preserved native physical directory layout (schema/ and tables/) in .amberdb archives for 1-to-1 extraction and portability.
* Added Tools->restore() with SHA-256 checksum verification, non-empty database safety checks, and automated deterministic binary index reconstruction via set_index.
* Enhanced Tools->all_tables() with dual scalar/list context (grouped hashref vs. flat list) and automated 4-digit year directory discovery (e.g. 2024/, 2025/, 2026/).
* Upgraded 'bin/convert_dbstore.pl' table discovery and added side-file reporting for .str string dictionaries alongside .del, .aut, and .cnt.
* Introduced CLI utility 'bin/amberdb_backup.pl' for command-line database dump and disaster recovery operations.
* Added comprehensive unit test suite t/amberdb_backup.t covering WAL streaming, .amberdb archiving, .dbase/.str preservation, and full database restore.
- Updated POD documentation across AmberDB, AmberDB::Transact, and AmberDB::Tools modules.
5.02 2026-08-25
- Initial public release prepared for CPAN and GitHub.
- High-performance Berkeley DB (DB_File) flat-file database engine.
- Packed 8-byte binary indexing pipeline for O(1) substr slicing.
- High-speed index-assisted full-text search with phonetic and language normalization.
- Tiered indexing (Active, Junk/Archived, and Hybrid AB/BA query modes).
- Multi-dimensional Columnar Facet indexing (.fac) with high-efficiency bitsets.
- Undo-journal transaction engine (transact_start, transact_end, transact_rollback) with automatic LIFO rollback.
- Multi-granularity concurrency control (table-level and record-level flock).
- Multilingual locale engine supporting 9 languages (en, tr, de, fr, es, ja, ru, ar, az) with Turkish dotless/dotted 'i' rules, case folding, and number/currency formatting.
- Cross-platform RAM-Disk helper and binary index converter CLI tools.
- Comprehensive test suite covering 37 test suites.
( run in 0.533 second using v1.01-cache-2.11-cpan-e7c6538aa59 )