view release on metacpan or search on metacpan
- Getter: $adb->config('simple') is retained as a transparent fallback and resolves to $self->{_cfg}->{use_simple}.
- Setter: $adb->config(simple => $val) automatically maps to $self->{_cfg}->{use_simple}.
- Constructor: AmberDB->new(cfg => { simple => 1 }) automatically normalizes input to { use_simple => 1 }.
* Updated test suite (t/amberdb_simple_mode.t) to assert single-key storage in _cfg and verify two-way getter/setter interoperability between 'simple' and 'use_simple'.
- [NAMESPACE CLEANUP & REMOVAL OF AMBIGUOUS METHOD ALIASES]
* Removed legacy typeglob aliases ('str' and 'short' in AmberDB::Date, 'by_code' and 'all' in AmberDB::Locale::Currency, and 'reindex' in AmberDB::Tools::Index) to prevent ambiguous method pollution on inherited AmberDB instances and elimin...
5.26.5 2026-10-10
- [CROSS-PLATFORM ARCHIVE RESTORE & DUMP SYSTEM]
* Refactored backup archive dump and restore (AmberDB::Tools::Maintain) to support cross-platform portable exports (.dumpdb, .dumpdel, .dumpaut, .dumpcnt) utilizing TSV and Base64 serialization, eliminating Berkeley DB endianness/binary inc...
* Rewrote restore() and dump() to strictly use encapsulated AmberDB table_write(), table_read(), recs_scan(), and table_close() methods, fully eliminating raw DB_File and tie/untie usage.
* Enhanced restore() argument parsing to accept single-string file path ($tools->restore('backup.amberdb')), hashref, or key-value pairs.
* Added automatic schema cache invalidation (delete $adb->{_table}->{$tid}) upon restoring tables to ensure subsequent schema introspection reloads cleanly.
* Added resilient error handling with eval surrounding index rebuilds during restore.
- [TABLE COUNT & LOCK PATH ROBUSTNESS]
* Added definedness guard for raw index key string in AmberDB::table_count.
* Standardized lock directory resolution via $self->path("lock_dir") in flock_open.
* Extended $adb->path() to support setter operations ($adb->path({ backup_dir => '...' }) and $adb->path(key => 'val')).
5.26.4 2026-10-01
- [IMPROVEMENTS]
* The reindex vacuum bug has been resolved.
* Some incorrect statements in "Changes" have been corrected.
5.26.3 2026-09-29
- [AUTOCOMPLETE & INSTANT SEARCH / AJAX QUERY SUGGESTION ENGINE (.ajw, .ajn)]
* Implemented high-performance dual-index query suggestion and autocomplete engine designed for Ajax instant-search bars and typeahead UI widgets.
* Introduced '.ajw' (Word Prefix Index): maps 2 to 12-character token prefixes to top candidate words with occurrence frequency scores (top 10 matches per prefix), supporting both native Unicode (Turkish/international alphabet) and ASCII-fo...
- [RELATIONAL PARENT/CHILD TABLES & MATCH INDEXING]
* Enforced transparent child table RAM-disk cache invalidation (_invalidate_child_ramdisk_tables) when parent_table receives writes or deletes.
* Validated match_block index construction and field_fetch O(1) lookups for child tables linked by parent_table (e.g., member_google).
- [ENCODER & BINARY BUFFER OPTIMIZATIONS]
* Optimized binary crop routines (t/amberdb_bin_crop.t) and range filtering algorithms in AmberDB::Base::Encoder and AmberDB::Array.
- [TOOLS REFACTOR & SUB-MODULE DECOMPOSITION (AmberDB::Tools)]
* Decomposed monolithic AmberDB::Tools (~2350 lines) into modular sub-modules under lib/AmberDB/Tools/:
- AmberDB::Tools::Index: Primary keys, full-text inverted search, columnar facets, match blocks, sort matrices, and URL slug indexing (set_index, set_readall, set_search, set_fields, set_filters, set_rwlnkall, set_sort, index_alltables, c...
- AmberDB::Tools::Maintain: Database maintenance, backup dump/restore, compacting, table deletion, and TSV/CSV conversion (dump, restore, vacuum, del_table, tie2csv, csv2tie, dir_tables, all_tables, _copy_file).
- AmberDB::Tools::Update: Table upgrades, schema migrations, and centralized update routines (update_table, update_all, replace_tablename, replace_blockdata, db_simple, convert_tables, update_storage, update_version).
* Preserved AmberDB::Tools as the unified facade wrapper inheriting from all sub-modules (use parent qw(...)) with 100% backward compatibility.
* Enforced strict zero-POD rule across all sub-modules; all user-facing documentation is consolidated in AmberDB::Tools.
- [CENTRALIZED CLI STORAGE & VERSION UPDATES (amberdb_cli.pl)]
* Integrated 'amberdb update storage' and 'amberdb update version' into bin/amberdb_cli.pl (and amberdb/amberdb.bat wrappers).
* Added CLI option parsing for --check, --force, --all, --no-backup, --tables, and --cpanm flags.
* Fully eliminated legacy bin/amberdb_setup.pl; all maintenance and provisioning tasks are routed through the main amberdb CLI.
* Registered bin/amberdb and bin/amberdb_cli.pl in Makefile.PL EXE_FILES.
- [RAM-DISK ZERO-FALLBACK & SHARED MEMORY ARCHITECTURE]
* Re-engineered RAM-disk mounting lifecycle using platform table mapping (%RM_DIR: Windows R:, Linux /dev/shm, macOS /Volumes).
* Fixed Windows ImDisk detection in AmberDB::Base::Ramdisk by appending a trailing slash ($rm_base/) in -d tests to resolve MSYS2/Windows drive root stat quirks.
* Enforced strict Zero-Fallback architecture: when unmounted or disconnected, ramdisk_dir, table_rdir, schema_rdir, and config_rdir are normalized strictly to "" (empty string) in both $cfg and {_path}.
* Completely removed shmem_dir and any fallback creation of dbstore/ramdisk directories on physical disk when unmounted; tables and routines never assume a RAM-disk is present.
* Removed table_rdir, schema_rdir, and config_rdir from default persistent @dirs initialization array in AmberDB::Base::set_datadir.
* Implemented pure RAM-disk Shared Memory API: get_shmem(), set_shmem(), del_shmem() with strict zero-fallback to disk when unmounted.
* Added candidate pruning for small survivor sets (<= 250 records) directly on the raw binary slice.
* Full lifecycle management for $blk:vals: automatically maintains and prunes values across sort_add, sort_modify, and sort_del.
- [FIELD QUERY ENHANCEMENTS]
* field_fetch now accepts named block arguments via resolve_block_idx().
* Fixed raw numeric value recovery for fallback queries on normalized .inx sort keys.
- [DATE ENGINE OPTIMIZATIONS]
* Added memoization caching to AmberDB::Date with rollover protection.
* Added reset_date() methods.
* Replaced direct hash access with fast accessor methods across dump, vacuum, and backup routines.
- [DIRECTORY ARCHITECTURE & LOCK PATH STANDARDIZATION]
* Standardized lock_dir and session_dir as single-key directories in AmberDB::Base::Ramdisk and AmberDB::Base.
* Removed obsolete fallback keys (lock_rdir) and legacy naming aliases (conf_dir, cfg_dir).
* In RAM-disk environments, lock_dir and session_dir are dynamically mounted under $ramdisk_dir/lock and $ramdisk_dir/session with 0755 permissions; when unmounted, they reside cleanly under $dbase_dir/lock and $dbase_dir/session.
* Aligned database lock directory structures with inter-process state locking specifications.
- [SCHEMA LOADING RESILIENCE]
* Wrapped 'do $target_path' inside eval { do $target_path } in AmberDB::Base::Schema (table_info() and dbase_info()).
* Prevents fatal process/worker termination when user .table or .dbase schema files contain syntax errors under global $SIG{__DIE__} handlers.
* Automated Timestamped Backups: Automatically backs up migrating tables as <table_name>-v<detected_ver>-<YYYY-MMDD>.db prior to rewriting.
* Authoritative Data Preservation:
- .unq (Unique & Synonym Dictionary): Identified as non-reconstructible authoritative master data; strictly exempted from derived index cleanup, preserved live, and snapshot-copied to <table_name>-v<ver>-<date>.unq.
- .del (Soft-Deleted Records Archive): Detected via exist_table($table, 'del'), backed up, and all archived records migrated to ABR v5.
- .aut (Audit Trail Log): Detected via exist_table($table, 'aut'), backed up, and user modification history migrated to ABR v5.
- .cnt (Read Counter): Detected via exist_table($table, 'cnt'), backed up, and live counter state preserved.
* Clean Re-indexing via insert_list: Cleans up all derived and legacy indexes (qw(inx src fld fac slg srt jinx jfld jsrc)) and rebuilds table data and indexes atomically through $adb->insert_list.
- [COMMAND LINE UTILITY] Automated Migration Script (bin/update_tables.pl):
* Added CLI migration script bin/update_tables.pl supporting --all, --table=<names>, --dbase=<dir>, --force, and --help options.
* Provides granular per-table progress reporting and comprehensive post-migration summary (tables processed, upgraded, already up-to-date, record counts, and companion file backups).
- [TRANSACTION SAFETY, SEARCH & CSV COMPATIBILITY]
* Transact Journal Escaping: Escaped \n, \r, \x1e, and \\ in binary record payloads logged to .txn journals, preventing binary length bytes from splitting WAL records or corrupting field boundaries.
* CSV Line Preservation: In tie2csv and vacuum, exported CSV records using _db_encode_legacy so exported files remain clean single-line human-readable text.
* Unindexed Search Word Extraction: Fixed search_table in unindexed mode to decode record fields before passing them to get_words, preventing binary length bytes from corrupting search tokens.
- [QUERY ENGINE & INDEX PERFORMANCE OPTIMIZATIONS] Adaptive Binary Intersect & Pure-Index Candidate Filtering:
* Adaptive Binary Search Pruning (bin_crop): Introduced dynamic thresholding between XS unpack probing and 8-byte aligned O(log N) binary search (substr). For massive posting lists (e.g. 600K records), eliminates huge Perl scalar allocation...
* Pure-Index Candidate Probing (search_table): Refactored multi-value/range filtering (e.g. 27-year date intervals) to evaluate candidate IDs directly against sorted 8-byte aligned index buffers via binary search, completely eliminating slo...
* Batch Unindexed Fallback: Replaced iterative table_readid loops in unindexed search fallbacks with single-pass read_list batching, eliminating repetitive open/close file descriptor syscalls.
- [TEST COVERAGE & PACKAGING]
* Added comprehensive test suite t/amberdb_update_table.t covering multi-era format decoding, update_table migration, backup naming, .del/.aut/.cnt/.unq handling, and update_all batch discovery.
* Updated MANIFEST to include bin/update_tables.pl and new test suites.
* All 47 test files (440 assertions) passing with 100% success rate.
5.23.2 2026-09-03
- [LOCALE ENGINE & MULTILINGUAL ARCHITECTURE] 10th Language - Global Base (gb) and Default Locale:
* Introduced Global Base (gb) as the 10th supported language and new universal default/fallback locale (replacing en).
* Implemented comprehensive multilingual Latin character preservation in alphabet_chars across European, Turkish, Nordic, French, German, Spanish, and Slavic-Latin alphabets.
* Added cross-lingual, accent-tolerant search regex mapping (regex_map) matching accented and unaccented variations (e.g. cafe matches café, munchen matches münchen, seker matches Åeker).
* Implemented canonical accent folding in accent_map for high-recall inverted search indexing (.src).
* Added lossless Unicode ligature conversion in ascii_map (Ã->ss, æ->ae, Å->oe, ı->i, ø->o, Å->l, Ä->d, ð->d, þ->th, É->e) for clean URL slug and ASCII ID generation.
* Updated main AmberDB abstract to "High-performance embedded NoSQL database engine for Perl".
5.23.0 2026-09-02
- [ARCHITECTURE & ID SIMPLIFICATION] Pure 64-bit Binary Engine & Deprecation of 8-Byte ASCII IDs:
* Deprecated 8-byte ASCII IDs (a8) across binary index structures in favor of pure 64-bit Big-Endian unsigned integer packing (Q>*), guaranteeing O(1) binary slicing with fixed 8-byte record strides.
* Simplified bin_encode() and bin_decode() by removing fragile \0 byte auto-detection and string unpacking heuristics.
* Removed legacy id_type schema attribute across all modules and test suites; standard relational tables strictly enforce positive integer numeric IDs.
- [STORAGE & PER-TABLE SIMPLE MODE] Hybrid Multi-Model Architecture with use_simple => 1:
* Introduced per-table use_simple => 1 attribute allowing key-value tables with arbitrary string keys up to 255 bytes (UUIDs, session tokens, emails, slugs, etc.) alongside standard relational tables.
* Preserved canonical physical table path ($dbase_dir/table/$table.db) for use_simple tables in standard database mode.
* Selective schema sanitization: tables with use_simple => 1 strip indexing, columnar, and caching definitions (blocks, match_block, search_block, facet_block, sort_block, record_index, use_cache, cache_ttl) to achieve zero index/cache I/O ...
* Enabled keep_deleted => 1 archiving (.del) on use_simple tables.
- [RDBM & INTEGRITY] Foreign Key and Cross-Table Isolation:
* Prohibited standard relational tables from binding foreign keys (RDBM) to use_simple tables in rdbm_target() and _resolve_field_value().
- [TESTS & VERIFICATION] Dedicated Test Coverage:
* Added comprehensive test suite t/amberdb_table_use_simple.t verifying arbitrary string keys, schema sanitization, zero index file creation, keep_deleted archiving, and RDBM isolation.
* Cleaned up legacy id_type occurrences across all 41 test files and concurrency stress tests (xt/amberdb_concurrency_stress.t).
- [DISTRIBUTION & METADATA] Decoupled Internal Module Versions and CPAN no_index:
* Stripped redundant $VERSION definitions from 15 internal/engine-only modules (Base, Cache, Transact, Index, Index::Facet, Index::Junk, Locale::Currency, Locale::Lang::*) to streamline release maintenance.
* Retained explicit $VERSION in public, standalone modules (AmberDB, AmberDB::Array, AmberDB::Date, AmberDB::Locale, AmberDB::Tools).
* Added comprehensive META_MERGE no_index configuration in Makefile.PL covering internal directories, namespaces, and packages.
5.22.0 2026-08-31
- [REFACTOR] Codebase Cleanliness:
* Removed redundant single-line helper is_rdbm_block() in favor of direct rdbm_target() calls across AmberDB::Index, AmberDB::Index::Junk, and AmberDB::Tools.
- [RDBM & INTEGRITY] Hardened Foreign Table Auto-Registration in field_to_list():
* Replaced decoupled table_autoid() and recs_put() sequence with atomic insert_id($target_table, 0, @record) to prevent ID race conditions.
* Corrected foreign record column alignment so auto-registered values are accurately placed at block index $target_blk rather than hardcoded column 1.
* Guaranteed that foreign table primary indexes (.inx) and secondary indexes (.unq, .src, .fac, .slg) are fully and consistently constructed upon auto-registration.
- [SECURITY & VALIDATION] Universal ID Verification and Simple Mode Key Sanitization:
* Eliminated opt-in config('id_check') requirement: every schema table now strictly enforces its schema id_type (positive integers for num, safe 8-byte chars for ascii).
* Enforced scalar ID requirement: any reference (ARRAY ref, HASH ref, etc.) passed as a record ID is strictly rejected across all modes.
* Introduced Safe Key Sanitization in Simple Mode: automatically trims leading/trailing whitespace, strictly rejects NUL bytes (\0) and control characters (\r, \n, \t, \x00-\x1F, \x7F) that corrupt Berkeley DB or CSV backups, and enforces a...
* Integrated unconditional id_check across table_autoid(), read_id(), modify_id(), delete_id(), and exist_id().
- [REFACTOR] Terminology and Codebase Refactoring: SEO -> Slug:
* Completely replaced SEO terminology with Slug across the entire codebase, test suite, and documentation without legacy aliases:
- Renamed methods: set_seourl() -> set_slug(), get_seourl() -> get_slug().
- Renamed schema attributes: seo_block -> slug_block, seo_max_len -> slug_max_len.
* Renamed test suite t/amberdb_seo_facet_bulk.t to t/amberdb_slug_facet_bulk.t.
- [STORAGE] Standardized URL Slug Map File Extension: .rwt -> .slg:
* Renamed binary slug map files from _0.rwt / _1.rwt to .slg (ID -> Slug) and .slg (Slug -> ID) across AmberDB, Index, Transact, and Tools.
* Updated Tools->set_index() to reconstruct .slg files with zero data loss.
- [DOCS] Caching Terminology Unification:
- Upgraded transaction engine specification to full ACID-Compliance with Strict Two-Phase Locking (Strict 2PL):
* Enforced Lock-Before-Write and Lock-Before-Read ordering across insert_id, modify_id, and delete_id for true serializable isolation.
* Introduced 'no_transact => 1' schema attribute and table_attr() support to exempt auxiliary tables from abort cascades while preserving LIFO rollback consistency.
- Added comprehensive ACID architectural guarantees section to documentation (README.md, Turkish and English User Guides).
- Clarified architectural distinction between high-throughput batch ETL imports and atomic business transactions.
- Standardized file open error diagnostics and OS-level reporting ($!) across all core modules:
* Added explicit OS error reporting ($!) to all open and tie failures in AmberDB, Base, Cache, Transact, and Tools.
* Replaced silent schema open failure in AmberDB::Base::table_write with diagnostic cluck and graceful return.
* Improved audit log error handling in AmberDB::auth_insert with cluck and record skipping.
- Redesigned 2-Pillar Disaster Recovery and Native Backup Architecture:
* Upgraded recs_back to continuous chronological time-series stream in 'backup/YYYY/YYYY-MM-DD.csv' eliminating folder clutter and ensuring zero-data-loss logging.
* Added Tools->dump() for creating portable, compressed '.amberdb' archives packaging schemas (schema/*.table, schema/*.dbase), authoritative data files (tables/*.db, tables/*.del, tables/*.aut, tables/*.cnt, tables/*_*.str), and cryptograp...
* Preserved native physical directory layout (schema/ and tables/) in .amberdb archives for 1-to-1 extraction and portability.
* Added Tools->restore() with SHA-256 checksum verification, non-empty database safety checks, and automated deterministic binary index reconstruction via set_index.
* Enhanced Tools->all_tables() with dual scalar/list context (grouped hashref vs. flat list) and automated 4-digit year directory discovery (e.g. 2024/, 2025/, 2026/).
* Upgraded 'bin/convert_dbstore.pl' table discovery and added side-file reporting for .str string dictionaries alongside .del, .aut, and .cnt.
* Introduced CLI utility 'bin/amberdb_backup.pl' for command-line database dump and disaster recovery operations.
* Added comprehensive unit test suite t/amberdb_backup.t covering WAL streaming, .amberdb archiving, .dbase/.str preservation, and full database restore.
- Updated POD documentation across AmberDB, AmberDB::Transact, and AmberDB::Tools modules.
5.02 2026-08-25
- Initial public release prepared for CPAN and GitHub.
- High-performance Berkeley DB (DB_File) flat-file database engine.
- Packed 8-byte binary indexing pipeline for O(1) substr slicing.
- High-speed index-assisted full-text search with phonetic and language normalization.
- Tiered indexing (Active, Junk/Archived, and Hybrid AB/BA query modes).
- Multi-dimensional Columnar Facet indexing (.fac) with high-efficiency bitsets.
- Undo-journal transaction engine (transact_start, transact_end, transact_rollback) with automatic LIFO rollback.
t/amberdb-locale_03_num2text.t
t/amberdb-locale_04_sort.t
t/amberdb-locale_05_formatting.t
t/amberdb-locale_06_currency.t
t/amberdb-locale_07_search_regex.t
t/amberdb-locale_08_word_normalization.t
t/amberdb-locale_09_gb.t
t/amberdb.t
t/amberdb_array.t
t/amberdb_array_sort.t
t/amberdb_backup.t
t/amberdb_bin_crop.t
t/amberdb_bin_ops.t
t/amberdb_cache.t
t/amberdb_cli.t
t/amberdb_connect_lifecycle.t
t/amberdb_date.t
t/amberdb_dbase_inheritance.t
t/amberdb_ecommerce_facet.t
t/amberdb_encapsulation.t
t/amberdb_escape_encode.t
MANIFEST.SKIP view on Meta::CPAN
# Version control
\B\.git\b
\B\.github\b
\B\.gitignore\b
\B\.gitattributes\b
# Temp, backup and editor files
~$
\.bak$
\.old$
\.tmp$
\.swp$
\#.*#$
\.\#.*
\b\.vscode\b
\b\.idea\b
\.code-workspace$
\blogs\b
\.log$
\bscratch\b
^\.backup\b
\b\.backup/
\b\.backup\b
\.rar$
\.zip$
\.7z$
^nul$
^NUL$
\bnul$
\bNUL$
[{}]
-scc_.*\.md$
^AGENTS\.md$
- **Ultra High-Performance**: Leverages Berkeley DB (`DB_File`) hash storage with $O(1)$ binary slicing and configurable in-memory buffers.
- **JOIN-Free JSON-like Extensible Block Records**: Eliminates complex relational SQL `JOIN` overhead by storing hierarchical, extensible block records. Newly added blocks and attributes are automatically indexed on the fly for low-latency multi-dime...
- **Schema-Driven Dynamic Runtime Manipulation**: Table-specific schemas govern field validations, encodings, and index mappings. Schemas and values are fully mutable and can be modified dynamically at runtime without requiring table recreation or mi...
- **8-Byte Packed Binary Indexing**: Primary and secondary indexes use unified 8-byte packed binary buffers (`Q>*`), enabling $O(1)$ substring slicing, sub-millisecond pagination, and memory-efficient `keys_only` scalar pipelines (with `use_simple =>...
- **Intelligent & Locale-Aware Accent Search**: Advanced full-text search engine (`.src`) equipped with regional language and accent intelligence, phonetic devoicing (`b/d/g -> p/t/k`), circumflex/accent unfolding (`â/î/û -> a/i/u`), apostrophe su...
- **Columnar Facet Indexing (`.fac`)**: High-performance multi-dimensional facet filtering with index-level bitwise intersections and bidirectional string dictionaries (`.unq`) for e-commerce, catalogs, and large categorical datasets.
- **Multi-Tier Junk & Lifecycle Management**: Segregates active records from historical/archived data (`.db` master vs `.jnk` tier) with seamless single-pass hybrid queries (`jnktype => 'A' | 'B' | 'AB' | 'BA'`).
- **ACID-Compliant Undo-Journal Transactions**: Full ACID multi-table transactions with disk-backed undo journaling (`dbstore/journal/txn_*`), Strict Two-Phase Locking (Strict 2PL), automatic LIFO rollback upon failure or abnormal process exit, and o...
- **2-Pillar Disaster Recovery & Native `.amberdb` Archiving**:
- **Pillar 1 (Continuous Recovery Stream):** Automatic append-only audit stream in `backup/YYYY/YYYY-MM-DD.csv` capturing every `insert`, `modify`, and `delete`.
- **Pillar 2 (Native Portable Archive):** Compressed, portable `.amberdb` archives containing schemas (`schema/*.table`, `schema/*.dbase`) and authoritative data files (`table/*.db`, `table/*.del`, `table/*.aut`, `table/*.cnt`, `table/*.unq`) with ...
- **Multi-Granularity Concurrency Control**: Non-blocking shared reads and exclusive writes at both table-level and individual record-level using OS-native `flock`.
- **ORM & Data Hydration (`inflate` / `deflate`)**: Native transformation between flat storage arrays and schema-mapped hash structures (`$adb->inflate` and `$adb->deflate`), including automatic RDBM foreign relationship resolution and repeating chil...
- **Granular Field Operations**: Direct field mutation without full record rewriting via `update_field`, positional child block insertion via `insert_field`, and safe targeted child deletion via `delete_field`.
- **Multilingual Locale Engine**: Out-of-the-box support for 10 languages (`gb` [default Global Base], `en`, `tr`, `de`, `fr`, `es`, `ja`, `ru`, `ar`, `az`) with language-specific case folding (e.g. Turkish `ı/I` and `i/İ`), cross-lingual accent fo...
- **High-Throughput 2-Phase Batch Operations**: High-performance batch ingestion pipeline (`insert_list`, `modify_list`, `delete_list`) opens master `.db` once for batch writing and executes single-pass index merging (`.inx`, `.src`, `.fld`, `.fac`, ...
- **Transparent Physical RAM-Disk Acceleration**: Integrated cross-platform orchestration (`tmpfs` Linux, `APFS` macOS, `ImDisk` Windows) across 4 operational tiers (0: Disk, 1: Hybrid Index-only, 2: Full RAM Mirror with dual-write, 3: Volatile pure ...
---
âââ table/ â Master Data & Derived Index Files
â âââ catalog_product.db â Primary key-value data table (DB_File Hash)
â âââ catalog_product.del â Soft-deleted records archive (keep_deleted)
â âââ catalog_product.aut â User audit trail log (log_owner)
â âââ catalog_product.cnt â View/hit counters (use_counter)
â âââ catalog_product.unq â Bidirectional string-to-ID dictionary
â âââ catalog_product.inx â Primary 8-byte packed ID and sort index
â âââ catalog_product.fld â Inverted exact-match field index ("$blk:$val")
â âââ catalog_product.src â Full-text keyword search index
â âââ catalog_product.fac â Columnar facet filter index
âââ backup/ â Disaster Recovery & Archives
âââ 2026/
âââ 2026-08-28.csv â Continuous time-series audit stream (Pillar 1)
âââ full_backup.amberdb â Compressed native database archive (Pillar 2)
```
### File Extension Reference
AmberDB classifies physical file extensions into three operational tiers:
#### Authoritative Master Data (Source of Truth â Non-Reconstructible)
- **`.db`** â **Primary Document Table**: Berkeley DB master document storage (`DB_File` Hash).
- **`.del`** â **Soft-Deleted Archive**: Historical archive of soft-deleted records (`keep_deleted`).
- **`.aut`** â **User Audit Trail**: Chronological user action and modification audit log (`log_owner`).
#### Derived Secondary Indexes (Reconstructible via `set_index`)
- **`.inx`** â **Primary & Sorted Record Index**: Binary array of active IDs, record count, highest ID, and pre-sorted order arrays (`sort_block`).
- **`.fld`** â **Inverted Match Index**: Block-level key-to-IDs inverted index (`match_block`).
- **`.src`** â **Full-Text Search Index**: Word-level token inverted index (`search_block`).
- **`.fac`** â **Facet Navigation Index**: Forward bitset index for faceted filter navigation (`facet_block`).
- **`.slg`** â **URL Slug Map**: Flat unified bidirectional slug map: `0:$rid` (IDâSlug) and `1:$slug` (SlugâID).
#### Runtime & Backup Files
- **`.amberdb`** â **Native Database Archive**: Compressed tar archive with schemas, data files, and SHA-256 manifest.
- **`.csv`** â **Continuous WAL Stream**: Daily chronological append-only audit stream (`backup/YYYY/YYYY-MM-DD.csv`).
- **`.cnt`** â **View / Hit Counter**: High-throughput concurrent atomic counter store (`use_counter`).
- **`.tmp`** â **Disk Buffer File**: Disk staging buffer file under `dbstore/buffer/` (`buffer_write`).
- **`.lock`** â **Process Mutex Lock**: OS `flock` process synchronization lock file.
---
## Installation
### Standalone Windows Executable (`amberdb.exe`)
}
```
### 7. Native Database Backup & Restore (`.amberdb`)
```perl
use AmberDB::Tools;
my $tools = AmberDB::Tools->new($adb);
# Create a compressed .amberdb backup archive
my $archive = $tools->dump();
# Archive created at: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb
# Restore archive with SHA-256 verification and automatic index rebuilding
my $result = $tools->restore(
file => "backup/2026/full_backup.amberdb",
force => 1, # Overwrite authorization for non-empty directories
reindex => 1 # Automatically reconstruct .inx, .src, .fld, .fac, .slg
);
```
### 8. High-Throughput Batch Operations (Batch ETL & Ingestion)
When importing or updating hundreds or thousands of records from CSV, JSON, or external APIs, use the 2-phase batch methods (`insert_list`, `modify_list`, `delete_list`). These methods open the master `.db` file once and update all secondary indexes ...
```perl
# Extract to C:\amberdb and add to User PATH (one-line PowerShell command):
[Environment]::SetEnvironmentVariable("Path", $env:Path + ";C:\amberdb", "User")
```
After restarting your terminal, use `amberdb` from any folder:
```cmd
amberdb setup
amberdb tables
amberdb read catalog_product 10
amberdb search catalog_product "laptop"
amberdb dump backup/catalog.amberdb
```
---
### 1. Unified Management CLI (`amberdb` / `amberdb.exe`)
The consolidated administrative CLI for infrastructure provisioning, database setup, RAM-disk orchestration, backups, and maintenance:
```bash
# Infrastructure setup (creates physical directories and baseline config)
amberdb setup
amberdb setup /path/to/dbstore
# RAM-disk management (Linux tmpfs, macOS APFS, Windows ImDisk)
amberdb setup ramdisk start 512M
amberdb setup ramdisk status
amberdb setup ramdisk stop
# Native backup (.amberdb dump and restore)
amberdb dump catalog file=backup/catalog.amberdb
amberdb restore file=backup/catalog.amberdb
# Storage layout & engine migrations
amberdb update storage --all
amberdb update version --check
# Re-index secondary binary indexes (.inx, .fld, .src, .slg)
amberdb reindex catalog
```
### 2. `bin/amberdb_daemon.pl` (Service Supervisor & Sync Daemon)
- [Komut Satırı Arayüzü (CLI) Kullanım Rehberi](wiki/TR-Guide-CLI.md)
- [AmberDB Veritabanı Sistemi & Mimari Rehberi](docs/TR.AmberDB_Veritabani_Sistemi.md)
- [AmberDB::Locale Kullanım Rehberi](docs/TR.AmberDB-Locale_Kullanim_Rehberi.md)
- [AmberDB vs SQL KarÅılaÅtırmalı Kullanım Rehberi](docs/TR.AmberDB-vs-SQL_Kullanim_Rehberi.md)
- [AmberDB vs SQLite Kıyaslama Raporu (600K Film)](docs/TR.AmberDB-vs-SQLite_Benchmark.md)
---
## Running Tests
AmberDB includes an exhaustive test suite covering core operations, indexing, transactions, search, facets, locales, and backups, along with multi-process concurrency stress tests:
```bash
# Run standard unit & integration test suite (66 test files, 650+ assertions)
prove -l t/
# Run author & extended integration test suite (RAM-disk & multi-process stress)
prove -l xt/
# or directly:
perl -Ilib xt/amberdb_ramdisk.t
perl -Ilib xt/amberdb_concurrency_stress.t
bin/amberdb_cli.pl view on Meta::CPAN
}
elsif ( $k =~ /^(?:force|f)$/ ) {
$opt_force = $val ? 1 : 0;
}
elsif ( $k =~ /^(?:check|c)$/i ) {
$method_args{check} = $val ? 1 : 0;
}
elsif ( $k =~ /^(?:all|a)$/i ) {
$method_args{all} = $val ? 1 : 0;
}
elsif ( $k =~ /^(?:no-backup|no_backup)$/i ) {
$method_args{no_backup} = $val ? 1 : 0;
}
elsif ( $k eq 'cpanm' ) {
$method_args{cpanm} = $v;
}
elsif ( $k eq 'manifest' ) {
$method_args{manifest} = $v;
}
elsif ( $k =~ /^(?:help|h|usage)$/i ) {
$opt_help = 1;
if ( defined $v && $v =~ /^(?:tr|en)$/i ) {
bin/amberdb_cli.pl view on Meta::CPAN
}
elsif ( $arg =~ /^(?:force|f)$/i ) {
$opt_force = 1;
}
elsif ( $raw =~ /^--?(?:check|c)$/i ) {
$method_args{check} = 1;
}
elsif ( $raw =~ /^--?(?:all|a)$/i ) {
$method_args{all} = 1;
}
elsif ( $raw =~ /^--?(?:no-backup|no_backup)$/i ) {
$method_args{no_backup} = 1;
}
elsif ( $raw =~ /^--?time$/i ) {
$opt_time = 1;
}
elsif ( !defined $opt_action ) {
$opt_action = $arg;
}
else {
push @pos_args, $arg;
bin/amberdb_cli.pl view on Meta::CPAN
amberdb setup [dir] # Initialize database skeleton in custom directory
amberdb setup ramdisk [start|stop|status] # RAM-disk drive management
Maintenance & Administrative Actions:
amberdb update storage [--check] [--force] # Directory & ABR v5 format migration
amberdb update version [--check] # MetaCPAN core version check / update
amberdb reindex products # Rebuild all derived indexes
amberdb check products # Physical file integrity check
amberdb vacuum products # Compact and reclaim BDB storage space
amberdb migrate products # Schema migration
amberdb export products file=backup.csv # Export table to CSV
amberdb import products file=new.csv # Import table from CSV
amberdb dump products file=backup.tar.gz # Create database backup archive
amberdb restore file=backup.tar.gz # Restore database from backup
amberdb rename from=old to=new # Rename table
amberdb drop temp_tbl force=1 # Permanently drop table and indexes
General Options:
--format=table|json|pretty|tsv|dumper Output format (Default: table. Or trailing: json, tsv, dumper)
--db=/path/to/dbstore Direct database directory without session
--token=TOKEN Active session token
--time, time, time=1 Prints execution time at the bottom
--dry-run Simulates execution without writes
--force Required confirmation for destructive actions
bin/amberdb_cli.pl view on Meta::CPAN
else {
# Central store under ~/.amberdb/<dbname>
$target_data_dir = "$AMBERDB_HOME/$conn_db";
}
$target_data_dir = resolve_abs_path($target_data_dir);
# Ensure target data directory and skeleton exist
make_path(
"$target_data_dir/table", "$target_data_dir/schema", "$target_data_dir/journal",
"$target_data_dir/lock", "$target_data_dir/session", "$target_data_dir/config",
"$target_data_dir/backup", "$target_data_dir/ramdisk"
);
my $conf_file = "$target_data_dir/config/core.conf";
if ( !-f $conf_file && open my $cfh, '>', $conf_file ) {
print $cfh "site_id $conn_db\nlanguage tr\n";
close $cfh;
}
$adb->set_datadir($target_data_dir);
$adb->{_connect}->{database} = $conn_db;
$adb->clear_cache('ramdisk');
bin/amberdb_cli.pl view on Meta::CPAN
print " amberdb connect eticaretim\n";
print " amberdb 1245 tables\n\n";
print " 3. Ãzel Klasör : 'ad\@yol' formatı ile oturum açılır\n";
print " amberdb connect eticaretim\@./dbstore\n";
print "=================================================================\n";
exit 0;
}
# If custom directory passed (e.g. amberdb setup /var/data or amberdb setup my_dir)
my $target_dir = resolve_abs_path($first_arg);
my @subdirs = qw(table schema journal lock session config backup ramdisk);
my @created;
my @existing;
make_path($target_dir) unless -d $target_dir;
for my $sub (@subdirs) {
my $dir_path = "$target_dir/$sub";
if ( -d $dir_path ) {
push @existing, $sub;
}
else {
make_path($dir_path);
bin/amberdb_cli.pl view on Meta::CPAN
}
$id //= 0;
my $data = delete $method_args{data};
# Ensure database directory and skeleton exist on write
my $db_dir = $adb->path('dbase_dir') || "dbstore";
if ( !-d $db_dir || !-d "$db_dir/table" ) {
make_path(
"$db_dir/table", "$db_dir/schema", "$db_dir/journal",
"$db_dir/lock", "$db_dir/session", "$db_dir/config",
"$db_dir/backup", "$db_dir/ramdisk"
);
my $conf_file = "$db_dir/config/core.conf";
if ( !-f $conf_file && open my $cfh, '>', $conf_file ) {
print $cfh "site_id amberdb\nlanguage tr\n";
close $cfh;
}
}
if ( !defined $data ) {
$data = scalar keys %method_args ? \%method_args : [ map { parse_value($_) } @pos_args ];
bin/amberdb_cli.pl view on Meta::CPAN
output_result( { status => 'ok', action => 'insert_id', table => $table, id => $res }, $opt_format );
exit 0;
}
# 9b. UPDATE_STORAGE
if ( $action eq 'update_storage' ) {
my %opts;
$opts{target_dir} = $adb->path('dbase_dir');
$opts{force} = $opt_force if $opt_force;
$opts{check} = $method_args{check} if exists $method_args{check};
$opts{no_backup} = $method_args{no_backup} if exists $method_args{no_backup};
$opts{tables} = $method_args{tables} // $method_args{table} // ( @pos_args ? join(',', @pos_args) : undef );
$opts{manifest} = $method_args{manifest} if exists $method_args{manifest};
my $res = $tools->update_storage(%opts);
output_result( $res, $opt_format ) if defined $opt_format;
exit 0;
}
# 9c. UPDATE_VERSION
if ( $action eq 'update_version' ) {
bin/commands.md view on Meta::CPAN
amberdb 1245 disconnect
```
### 4. Bakım ve Yönetim Eylemleri
```bash
amberdb reindex products
amberdb check products
amberdb vacuum products
amberdb export products file=products.csv
amberdb import products file=products.csv
amberdb dump products file=backup.tar.gz
amberdb restore file=backup.tar.gz force=1
amberdb rename from=old_table to=new_table
amberdb drop temp_table --force
```
docs/EN.About_AmberDB.md view on Meta::CPAN
[Home](index.html) ⢠[About](EN.About_AmberDB.html) ⢠[Quick Start](index.html#quick-start) ⢠[Tutorial](EN.AmberDB_User-Guide.html) ⢠[Benchmark](EN.AmberDB-vs-SQLite_Benchmark.html) â¢&...
---
# About
**AmberDB** is an embedded NoSQL database engine for Perl that processes Array-based records supporting JSON-like nested and complex structures. Thanks to custom indexes generated through schema definitions, it performs block matching, querying, full...
It is built on the C code of Berkeley DB (`DB_File`) provided by the standard Perl package and does not use any other database engine. `DB_File` is Berkeley DB's low-level interface that provides key-value access via Perl. AmberDB constructs a data m...
---
## Why AmberDB?
AmberDB brings together diverse capabilities within a single Perl database engine:
* **No External Database Server:** Embedded directly into the application as an in-process object.
* **Array-Based Records:** Database records are Arrays (lists) supporting JSON-like nested structures.
* **Full CRUD Operations:** Fast, direct insert, read, update, and delete methods.
docs/EN.About_AmberDB.md view on Meta::CPAN
$adb->modify_list("products", @records); # batch modify
$adb->delete_list("products", @ids); # batch delete
```
For instance, when inserting a list of one thousand records using `insert_list`, AmberDB creates the records and their indexes in a single batch process. Similarly, a massive price list can be modified within a single batch operation. This makes batc...
---
## Portable Backups
Because AmberDB is file-based, it is fully portable. It also includes safer native database tools for backup and restoration.
A database can be exported into a portable `.amberdb` archive containing the authoritative database state and schema. Integrity is verified using SHA-256, and derived indexes are rebuilt during restoration. This makes it possible to migrate or archiv...
---
## Epilogue
AmberDB provides an experience close to **PostgreSQL + Elasticsearch + Redis**, achieving this at an extremely low footprint and cost. It requires no standalone server installation and embeds directly into your application. It consumes minimal system...
AmberDB is released under the **Artistic License 2.0**. Source code, documentation, and examples are available and actively maintained on CPAN and GitHub.
* **GitHub Issues:** [https://github.com/marufcetin/amberdb/issues](https://github.com/marufcetin/amberdb/issues)
docs/EN.AmberDB-vs-SQL_User-Guide.md view on Meta::CPAN
Common tasks that require external libraries, database triggers, or external cache daemons in SQL stacks are natively built into AmberDB:
| Feature | Standard Solution in SQL | Built-in AmberDB Solution |
|---|---|---|
| **Automatic SEO URL Slugs** | Custom slugify code, database uniqueness check queries. | `slug_block => [1]` automatically creates collision-free slugs like `/product/sony-wh-1000xm5` upon insert/update (`get_slug`). |
| **User Audit Trail (Audit Log)** | Separate audit tables, database triggers, or ORM event hooks. | `log_owner => 1` logs every modification into `.aut`. Call `$adb->auth_view("table", $id)` for an instant HTML timeline. |
| **High-Speed In-Memory Caching & Ephemeral Data (RAM-Disk)** | Installing and managing external caching servers (`Redis` or `Memcached`) to bypass disk I/O bottlenecks, managing TCP network overhead and cache-sync logic. | Zero external servers or ...
| **Safe Soft-Delete** | Adding `is_deleted` column and remembering `WHERE is_deleted = 0` on every query. | `keep_deleted => 1` moves deleted records to a `.del` archive. Prevents data leaks with zero query filtering overhead. |
* **Reference:** [User Guide Section 12: URL Slugs](EN.AmberDB_User-Guide.html#12-automated-url-slug-management) · [Section 13: Shared RAM Cache](EN.AmberDB_User-Guide.html#13-unified-shared-ram-cache-db--inx-and-buffer) · [Section 17: Audit Trail]...
---
## 10. Quick Reference Cheat Sheet
A side-by-side mapping for common database operations:
| SQL Statement | AmberDB Method | AmberDB Usage Example |
|---|---|---|
| `INSERT INTO t VALUES (...)` | `insert_id` | `$id = $adb->insert_id("t", 0, @fields);` |
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
7. [Transaction Safety, ACID Guarantees, and Crash Recovery (Transactions)](#7-transaction-safety-acid-guarantees-and-crash-recovery-transactions)
8. [High-Throughput Batch Operations (Batch ETL & Ingestion)](#8-high-throughput-batch-operations-batch-etl--ingestion)
9. [Schema Configuration (.table & In-Memory)](#9-schema-configuration-table--in-memory)
10. [Database Group Structure (.dbase)](#10-database-group-structure-dbase)
11. [Smart Tiered (Hot / Cold Junk) Indexing](#11-smart-tiered-hot--cold-junk-indexing)
12. [Automated URL Slug Management](#12-automated-url-slug-management)
13. [Transparent Physical RAM-Disk Acceleration & In-Memory Storage](#13-transparent-physical-ram-disk-acceleration--in-memory-storage)
14. [Configuration and Deterministic Flag Management (`config`)](#14-configuration-and-deterministic-flag-management-config)
15. [Data Structures, Low-Level Table and Stream Operations](#15-data-structures-low-level-table-and-stream-operations)
16. [Faceted Search & Category Filters (Facet Engine)](#16-faceted-search--category-filters-facet-engine)
17. [User Audit Trail and Backup](#17-user-audit-trail-and-backup)
18. [Maintenance and Repair Tools (AmberDB::Tools)](#18-maintenance-and-repair-tools-amberdbtools)
19. [File Extensions Map](#19-file-extensions-map)
20. [Directory Structure](#20-directory-structure)
21. [Developer Best Practices and Recommendations](#21-developer-best-practices-and-recommendations)
22. [Full Working Example (Checkout & Stock Transaction Scenario)](#22-full-working-example-checkout--stock-transaction-scenario)
23. [Method Quick Reference Table](#23-method-quick-reference-table)
24. [Why Use AmberDB? (Comparison with SQL and SQLite)](#24-why-use-amberdb-comparison-with-sql-and-sqlite)
25. [Boundaries and Debated Topics (Physical Constraints vs. Conscious Architectural Choices)](#25-boundaries-and-debated-topics-physical-constraints-vs-conscious-architectural-choices)
---
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
---
### 5.2 Flexible & Arbitrary Record IDs (No 8-Byte Limit)
The standard mode **8-byte limit** and **strict ASCII/numeric format constraints** are relaxed in Simple Mode (`id_check` accepts arbitrary scalar keys and applies safe key sanitization):
- **Emails and Special Characters:** `user@example.com`, `api:v1:user:1005`
- **Long Tokens and UUIDs:** `sess_99999_abcdef_1234567890_extra_long_token` (up to 255 bytes)
- **Hyphenated Codes and Prefixes:** `TR-2026-08-31-INVOICE-001`
- **Unicode / Multilingual Keys:** `prod_özellik_kırmızı_xl`
- **Safe Key Sanitization:** Automatically trims leading/trailing whitespace (`trim_space`); strictly rejects NUL bytes (`\0`), control characters (`\r`, `\n`, `\t`), and references (ARRAY/HASH refs) to protect Berkeley DB C layers and CSV backup int...
- **Auto-ID Flexibility:** Custom IDs are not constrained to be strictly greater than `lastid`.
```perl
$adb->insert_id( 'sessions', 'user@example.com', 'Active', 'Chrome', time() );
my @sess = $adb->read_id( 'sessions', 'user@example.com' );
```
---
### 5.3 Data Operations (CRUD & Bulk)
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
if ($@) {
$adb->transact_error( 'sessions', $@ );
}
my $txn = $adb->transact_end(); # Auto-rollbacks on error, commits if clean
```
---
### 5.6 Continuous Daily Backup Logs (`recs_back`)
Because text backup is schema-independent, **daily audit and continuous recovery streaming (`recs_back`)** is fully active in Simple Mode.
In accordance with Simple Mode's flat directory structure, no separate `backup/` or `YYYY/` subfolder is created. Every `insert_id` (`add`), `modify_id` (`edit`), and `delete_id` (`del`) operation is logged directly to **`$dbase_dir/YYYY-MM-DD.csv`**...
```text
2026-08-31 14:30:00 admin add sessions sess_token_99999 Active\x1f192.168.1.50
2026-08-31 14:31:15 admin edit sessions sess_token_99999 Closed\x1f192.168.1.50
2026-08-31 14:32:00 admin del sessions sess_token_99999
```
- To disable backup logging for volatile caches, configure `cfg => { no_backup => 1 }` or `$adb->config(no_backup => 1)`.
- Custom backup targets can be set via `path => { backup_dir => "/custom/backup/path" }`.
---
### 5.7 RAM-Disk Architecture & Caching in Simple Mode
In standard mode, AmberDB manages RAM-disk staging via schema `use_ramdisk => 2` rules.
**In Simple Mode, RAM-disk utilization is direct and flexible:**
Since Simple Mode requires no schema files, creating a high-performance in-memory cache or session store simply involves binding a second AmberDB instance directly to the RAM-disk / tmpfs mount:
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
# 1. Persistent disk instance (For durable storage)
my $db_disk = AmberDB->new(
path => { dbase_dir => "/var/data/app/dbstore/table" },
cfg => { use_simple => 1 },
);
# 2. RAM-Disk instance (Zero-latency in-memory cache/session store)
# (Linux: /dev/shm or tmpfs, Windows: ImDisk, macOS: APFS RAM-Disk /Volumes/AmberDB_RAM)
my $db_ramdisk = AmberDB->new(
path => { dbase_dir => "/dev/shm/amberdb_cache" },
cfg => { use_simple => 1, no_backup => 1 }, # Disable backup for pure transient cache
);
# In-memory reads and writes at nanosecond speed:
$db_ramdisk->insert_id( "sessions", $session_token, $user_id, time() );
my @sess = $db_ramdisk->read_id( "sessions", $session_token );
```
Benefits of this dual-instance design:
- In-memory tables run without disk I/O bottlenecks.
- Persistent tables remain safely on durable physical storage.
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
| **Schema Files (`.table`, `.dbase`)** | Required & Enforced | None / Schemaless |
| **Arbitrary & Long Record IDs** | 8-Byte / Strict ASCII Limits | **Completely Unrestricted** |
| **Direct CRUD (`insert_id`, `read_id`)** | $O(1)$ | **$O(1)$ (Max Throughput)** |
| **Bulk Operations (`insert_list`, etc.)** | Supported | Supported |
| **Table Scan (`read_all`)** | Binary `.inx` or Direct | Direct Streaming Scan |
| **Pagination (`limit`) & `keys_only`** | Supported | Supported |
| **In-Memory Sorting (`sort => 2`)** | Supported | Supported |
| **Field Matching (`field_fetch`)** | Indexed `.fld` $O(1)$ | Sequential Streaming Scan |
| **Word Search (`search_table`)** | Inverted Index `.src` | Collation Streaming Scan |
| **ACID Transactions (`transact_*`)** | Supported (Index Undo) | **Supported (Raw Undo)** |
| **Continuous Daily Backup (`recs_back`)** | Supported (`backup/YYYY/`) | **Supported (Same Directory `YYYY-MM-DD.csv`)** |
| **Secondary Indexes (`.inx, .fld, .src, .slg, .fac`)** | Generated & Maintained | **Disabled (Zero Index Cost)** |
| **URL Slug Mapping (`.slg`)** | Auto Generated | Disabled |
| **Audit Logs (`.aut`) & Archive (`.del`)** | Schema-Driven | Disabled |
| **Directory Hierarchy** | `table/`, `schema/`, `backup/`, etc. | **Flat Single Directory (`$dbase_dir/<table_name>.db`)** |
| **Secondary Indexes (`.inx, .fld, .src, .slg, .fac`)** | Generated & Maintained | **Disabled (Zero Index Cost)** |
| **URL Slug Mapping (`.slg`)** | Auto Generated | Disabled |
| **Audit Logs (`.aut`) & Archive (`.del`)** | Schema-Driven | Disabled |
| **Directory Hierarchy** | `table/`, `schema/`, `backup/`, etc. | **Flat Single Directory (`$dbase_dir/<table_name>.db`)** |
---
## 6. Indexing and Search Engine
AmberDB maintains structured binary index files based on the schema configuration.
### 6.1 Index Types
| Extension | Index Type | Description |
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
### 9.1 Database and Table Directory Layout
AmberDB stores tables, indexes, and schema definitions in dedicated physical directories under the configured `dbstore` root:
| Directory | Purpose |
|---|---|
| `dbstore/table/` | Base data (`.db`) and binary indexes (`.inx`, `.fld`, `.src`, `.fac`, `.slg`) |
| `dbstore/schema/` | Schema files (`.table`) and group configs (`.dbase`) |
| `dbstore/config/` | Plain-text `.conf` configuration files, `connect.pl` authentication profile |
| `dbstore/backup/` | Daily CSV audit backups (`dbgun/YYYYMMDD/`) |
| `dbstore/journal/`| Transaction undo WAL logs and Tier 4 write-behind queues (`txn_*`, `sync_ramdisk`) |
| `dbstore/lock/` | Process and table-level `flock` lock files when RAM-disk is unmounted (`*.lock`) |
| `dbstore/session/`| Process session token files when RAM-disk is unmounted (`*.sess`) |
> [!NOTE]
> **RAM-Disk Directory Structure:** When an OS RAM-disk is mounted (e.g. `R:/amberdb_$dbname` on Windows, `/dev/shm/amberdb_$dbname` on Linux, `/Volumes/amberdb_$dbname` on macOS), subdirectories (`table/`, `schema/`, `config/`, `lock/`, `session/`, ...
> [!IMPORTANT]
> **Directory Structure Compatibility Note:** The only manual action required when upgrading legacy projects is to rename your database directory's `dbstore/scheme/` folder to **`dbstore/schema/`**. All programmatic path resolutions and API calls are...
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
| `use_counter` | `0 / 1` | `0` | `usecnt` | Enables automated hit/view read counters in `.cnt` files. |
| `parent_table` | `string` | `""` | - | Parent table name for vertical partitioning (child table shares the same primary ID). |
| `force` | `0 / 1` | `0` | - | When `1`, `insert_id` overwrites existing records rather than failing (Replace mode). |
| `min_char` | `integer` | `2` | `minchar` | Minimum word length for full-text search indexing (1, 2, or 3). |
| `stop_word` | `string` | `""` | `nextkey` | Stop-words excluded from full-text search indexing (e.g., `"the and for with"`). |
| `repeat_ids` | `integer` | `undef` | - | Target block number where extracted child item IDs are consolidated. |
| `repeat_start` | `integer` | `undef` | - | Starting block index for dynamic repeating child rows (order items, cart lines). |
| `view_block` | `ARRAY` | `[]` | - | Priority block numbers displayed in UI / CMS listing views. |
| `use_menu` | `0 / 1` | `1` | - | Controls display of the table in admin panel navigation menus. |
| `no_transact` | `0 / 1` | `0` | - | Exempts table from transactional rollback error propagation. |
| `no_backup` | `0 / 1` | `0` | - | Disables daily CSV user audit logging for this table. |
---
### 9.7 Block (Field) Definitions, 8 Core Field Types, UI Inputs, and Validation Reference
Each block definition inside the `blocks` array supports the following attributes:
#### 9.7.1 Core Block Attributes
| Attribute | Type | Description | Example |
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
---
## 14. Configuration and Deterministic Flag Management (`config`)
Runtime behavior can be tuned and safely configured via the `$adb->config()` method:
```perl
# Bulk or single configuration assignment (Recommended)
$adb->config(
no_write => 1, # Read-only maintenance mode: block all writes
no_backup => 1, # Disable daily CSV audit logging for all tables
use_simple => 1, # Direct unindexed mode: bypasses secondary index generation
keys_only => 1, # read_all returns IDs only
ramdisk_size => '1024M', # RAM-Disk (tmpfs/APFS/ImDisk) size (Default: 512M)
);
# Single scalar getter:
my $no_write = $adb->config('no_write');
# Bulk getter (returns a safe shallow copy):
my $cfg = $adb->config();
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
```perl
# Retrieve user audit history as formatted HTML
my $history_html = $adb->auth_view("catalog_product", 5001);
print $history_html;
# Output:
# add 2026-08-14 10:15 admin_user
# edit 2026-08-14 11:30 editor_user
```
### 17.2 Continuous Recovery Stream (`YYYY-MM-DD.csv`)
AmberDB automatically appends every `insert`, `modify`, and `delete` operation into a clean, chronological time-series stream in `backup/YYYY/YYYY-MM-DD.csv`.
Each entry is tab-separated (`\t`) using the standard format:
`[Timestamp] \t [User] \t [Action] \t [Table] \t [Record ID] \t [Packed Values]`
To disable this backup stream:
* **In Table Schema (Per-Table):** Add `no_backup => 1` in the table schema to disable logging for that specific table only.
* **Globally via Config (All Tables):** Set `$adb->config(no_backup => 1);` to disable logging across all tables.
### 17.3 Native Database Archive (`.amberdb` Dump & Restore)
AmberDB packages all schemas (`schema/*.table`, `schema/*.dbase`) and authoritative data files (`tables/*.db`, `tables/*.del`, `tables/*.aut`, `tables/*.cnt`) alongside cryptographically verified SHA-256 checksums in a single compressed, portable **`...
Derived index files (`.inx`, `.src`, `.fld`, `.fac`) are intentionally excluded to keep archives compact and ensure future-proof portability; `restore` deterministically rebuilds all indexes via `set_index`.
```perl
use AmberDB;
use AmberDB::Tools;
my $adb = AmberDB->new(path => { dbase_dir => "./dbstore" });
my $tools = AmberDB::Tools->new($adb);
# 1. Create full database backup archive (.amberdb)
my $archive = $tools->dump();
# Output: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb
# 2. Export specific tables as a focused snapshot archive
$tools->dump(
file => "backup/2026/catalog_backup.amberdb",
tables => ["catalog_product", "catalog_category"]
);
# 3. Restore database archive and automatically rebuild all indexes
$tools->restore(
file => "backup/2026/catalog_backup.amberdb",
force => 1, # Overwrite confirmation for non-empty target directories
reindex => 1 # Automatically reconstruct binary indexes from source data
);
```
#### CLI Command-Line Utility (`bin/amberdb_backup.pl`)
```bash
# Dump entire database to default archive
perl bin/amberdb_backup.pl --dump --file backup/2026/full_backup.amberdb
# Dump specific tables only
perl bin/amberdb_backup.pl --dump --tables products,orders
# Restore database archive with integrity checks and automatic reindexing
perl bin/amberdb_backup.pl --restore --file backup/2026/full_backup.amberdb --force
```
---
## 18. Maintenance and Repair Tools (AmberDB::Tools)
`AmberDB::Tools` provides utilities for reindexing, table vacuuming, and data migration:
```perl
use AmberDB;
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
â âââ catalog_product.src â Full-text search index
â âââ catalog_product.fac â Facet filtering index
â âââ catalog_product.unq â String dictionary
â âââ catalog_product.slg â Bidirectional URL slug map ("0:$id", "1:$slug")
â âââ catalog_product.aut â Audit trail
â âââ catalog_product.del â Soft-deleted records
âââ ramdisk/ â RAM-Disk Mount & Storage (Linux tmpfs, macOS APFS, Windows ImDisk)
âââ buffer/ â Transient Disk Buffer / Staging Files
âââ journal/ â Active Transaction Journals (txn_*)
âââ pids/ â Lock Files
âââ backup/ â Daily CSV Backups
```
---
## 21. Developer Best Practices and Recommendations
1. **Use `insert_list` for Bulk Ingestion:** When adding hundreds of records, use `insert_list` instead of looping over `insert_id`. Batch mode writes all records in a single file session and rebuilds indexes in one pass.
2. **Wrap Multi-Step Writes in `transact_start`:** Always wrap inventory deductions, checkout sequences, or multi-table balance updates inside transactions.
3. **Index Only Required Fields:** Only assign fields to `match_block` or `search_block` if they are actively queried to minimize disk write overhead.
4. **Always Handle Pagination Return Signatures Correctly:** When passing `$limit > 0` to `read_all`, `field_fetch`, or `search_table`, remember that the first returned value is `$total_count` integer. Never unpack into a single array (`my @records =...
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
### 24.4 Direct Inverted Key Lookups (Zero Query Planner Overhead)
In SQL, running `SELECT id FROM orders WHERE customer_id = 'A'` requires parsing, query plan evaluation, cost optimization, and virtual machine execution.
In AmberDB, `field_fetch` is a direct hash key lookup on Berkeley DB returning packed binary buffers. Query planning overhead is zero.
### 24.5 Built-in Lifecycle and Domain Features
- **Automatic URL Slug Management:** When titles or categories change, clean slugs like `/products/laptop-pro-m3` and conflict resolution suffixes are generated automatically.
- **Audit Trails (.aut):** User identity, action type (`add`, `edit`, `del`), and timestamps are recorded without extra tables.
- **Safe Soft Deletion (.del):** Deleted records are archived safely and can be inspected or restored.
- **Zero Configuration & Portability:** Copying the database directory creates a complete, standalone backup that can run on any Perl-enabled system.
---
## 25. Boundaries and Debated Topics (Physical Constraints vs. Conscious Architectural Choices)
In database design, every architectural decision serves a specific optimization goal. Certain characteristics that developers coming from traditional SQL environments might initially perceive as "constraints" or "omissions" are, in fact, **deliberate...
### 25.1 Physical and Environmental Boundaries (Out-of-Scope Scenarios)
The following scenarios lie outside the intended operational scope of an embedded, file-based database engine like AmberDB:
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
}
my $txn = $adb->transact_end(); # Hata varsa otomatik rollback, yoksa commit
```
---
### 5.6 Günlük Sürekli Yedekleme Günlükleri (Daily Backup Logs - `recs_back`)
Basit mod Åemadan baÄımsız olduÄundan, **metin tabanlı sürekli denetim ve kurtarma akıÅı (`recs_back`)** basit modda da varsayılan olarak devrededir.
Basit modun düz dizin yapısı gereÄi ayrı bir `backup/` veya `YYYY/` alt klasörü oluÅturulmaz; yapılan her `insert_id` (`add`), `update_id` (`edit`) ve `delete_id` (`del`) iÅlemi doÄrudan tablolarla aynı dizinde bulunan **`$dbase_dir/YYYY-...
```text
2026-08-31 14:30:00 admin add sessions sess_token_99999 Aktif\x1f192.168.1.50
2026-08-31 14:31:15 admin edit sessions sess_token_99999 Kapali\x1f192.168.1.50
2026-08-31 14:32:00 admin del sessions sess_token_99999
```
- İstenirse `cfg => { no_backup => 1 }` veya `$adb->config(no_backup => 1)` ile yedekleme günlükleri devre dıÅı bırakılabilir.
- Ãzel bir yedek dizini tanımlanmak istendiÄinde `path => { backup_dir => "/harici/yedek/yolu" }` verilebilir.
---
### 5.7 Basit Modda RAM-Disk Mimarisi ve Ãnbellekleme
AmberDB standart modda RAM-disk hızlandırmasını Åemadaki `use_ramdisk => 2` kuralı ile yönetir.
**Basit modda ise RAM-disk kullanımı çok daha doÄrudan ve esnektir:**
Basit mod Åemaya ihtiyaç duymadıÄından, yüksek performanslı bir bellek önbelleÄi / geçici oturum deposu oluÅturmak için ikinci bir AmberDB basit nesnesi doÄrudan RAM-disk yoluna baÄlanır:
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
# 1. Kalıcı disk nesnesi (Kalıcı veriler için)
my $db_kalici = AmberDB->new(
path => { dbase_dir => "/var/data/eticaret/dbstore/table" },
cfg => { use_simple => 1 },
);
# 2. RAM-Disk nesnesi (Sıfır gecikmeli hızlı oturum/önbellek tabloları için)
# (Linux: /dev/shm veya tmpfs, Windows: ImDisk, macOS: APFS RAM-Disk /Volumes/AmberDB_RAM)
my $db_ramdisk = AmberDB->new(
path => { dbase_dir => "/dev/shm/amberdb_cache" },
cfg => { use_simple => 1, no_backup => 1 }, # Ãnbellek için yedekleme kapatılabilir
);
# RAM üzerinde nanosaniye hızında oturum okuma/yazma:
$db_ramdisk->insert_id( "oturumlar", $session_token, $user_id, time() );
my @oturum = $db_ramdisk->read_id( "oturumlar", $session_token );
```
Bu çift nesneli mimari sayesinde:
- RAM-disk üzerindeki tablolar disk I/O darboÄazına takılmadan bellek hızında çalıÅır.
- Kalıcı tablolar ana depolama alanında güvenle tutulmaya devam eder.
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
| **Åema Dosyaları (`.table`, `.dbase`)** | Zorunlu / Kullanılır | Yok / Åemasız |
| **Ãzel ve Uzun ID'ler (UUID, E-posta, vb.)** | 8 Bayt / ASCII Kısıtlı | **Tamamen Serbest** |
| **Tekil CRUD (`insert_id`, `read_id`, vb.)** | $O(1)$ | **$O(1)$ (Maksimum Hız)** |
| **Toplu İÅlemler (`insert_list`, vb.)** | Desteklenir | Desteklenir |
| **Tüm Tablo Okuma (`read_all`)** | `.inx` veya doÄrudan | DoÄrudan Dosya Taraması |
| **Sayfalama (`start`/`limit`) ve `keys_only`** | Desteklenir | Desteklenir |
| **Bellek İçi Sıralama (`sort => 2`)** | Desteklenir | Desteklenir |
| **Alana Göre Filtre (`field_fetch`)** | `.fld` İndeksli $O(1)$ | Sıralı Dosya Taraması |
| **Tam Metin Arama (`search_table`)** | `.src` Ters İndeksli | Sıralı Dosya Taraması (Türkçe Normalizasyonlu) |
| **ACID İÅlemler (`transact_*`)** | Desteklenir (İndeks Geri Alma Dahil) | **Desteklenir (Ham Veri Geri Alma)** |
| **Sürekli Yedekleme AkıÅı (`recs_back`)** | Desteklenir (`backup/YYYY/`) | **Desteklenir (Aynı Dizinde `YYYY-MM-DD.csv`)** |
| **İkincil İndeksler (`.inx, .fld, .src, .fac`)** | OluÅturulur ve Güncellenir | **OluÅturulmaz (Sıfır İndeks Maliyeti)** |
| **URL Slug Rewrite (`.slg`)** | Otomatik Ãretilir | Devre DıÅı |
| **Denetim İzi (`.aut`) ve ArÅiv (`.del`)** | Åema Kuralına Göre Tutulur | Devre DıÅı |
| **Dizin Yapısı** | `tables/`, `schema/`, `backup/` vb. | **Düz Kök Dizin (`$dbase_dir/<tablo>.db`)** |
| **İkincil İndeksler (`.inx, .fld, .src, .fac`)** | OluÅturulur ve Güncellenir | **OluÅturulmaz (Sıfır İndeks Maliyeti)** |
| **URL Slug Rewrite (`.slg`)** | Otomatik Ãretilir | Devre DıÅı |
| **Denetim İzi (`.aut`) ve ArÅiv (`.del`)** | Åema Kuralına Göre Tutulur | Devre DıÅı |
| **Dizin Yapısı** | `tables/`, `schema/`, `backup/` vb. | **Düz Kök Dizin (`$dbase_dir/<tablo>.db`)** |
---
## 6. İndeksleme ve Arama Mekanizması
AmberDB, tablolara hızlı eriÅim saÄlamak için veriyi Åemada tanımlanan kurallara göre ikili (binary) indeks dosyalarına yazar.
### 6.1 İndeks Türleri
| Dosya Uzantısı | İndeks Türü | Açıklama |
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
### 9.1 Veritabanı ve Tablo Dizin Yapısı
AmberDB tabloları, Åemaları ve geçici/kalıcı dosyaları, belirlenen `dbstore` ana veri dizini altında fiziksel klasörlere ayrılarak saklanır:
| Dizin | Görevi |
|---|---|
| `dbstore/table/` | Kalıcı `.db` ana veri, `.inx` kayıt ve sıralama indeksi, `.fld` eÅleÅtirme, `.src` arama, `.fac` facet, `.slg` slug dosyaları |
| `dbstore/schema/` | Kalıcı `.table` tablo Åemaları ve `.dbase` grup yapılandırma dosyaları |
| `dbstore/config/` | Kalıcı `.conf` düz metin ayar ve konfigürasyon dosyaları, `connect.pl` profil dosyası |
| `dbstore/backup/` | Günlük CSV denetim yedekleri (`dbgun/YYYYMMDD/`) |
| `dbstore/journal/`| İÅlem geri alma (WAL) ve Tier 4 gecikmeli yazma günlükleri (`txn_*`, `sync_ramdisk`) |
| `dbstore/lock/` | RAM-disk baÄlı deÄilken süreç ve tablo seviyesi kilit dosyaları (`*.lock`) |
| `dbstore/session/`| RAM-disk baÄlı deÄilken süreç oturum dosyaları (`*.sess`) |
> [!NOTE]
> **RAM-Disk Dizin Yapısı:** İÅletim sisteminde RAM-disk baÄlandıÄında (Windows'ta `R:/amberdb_$dbname`, Linux'ta `/dev/shm/amberdb_$dbname`, macOS'ta `/Volumes/amberdb_$dbname`), RAM-disk kök dizini altında dinamik olarak `table/`, `schema...
> [!IMPORTANT]
> **Dizin Yapısı Uyumluluk Notu:** Eski projelerden yükseltme yaparken yapmanız gereken tek fiziksel iÅlem; veritabanı dizininizdeki `dbstore/scheme/` klasörünün adını **`dbstore/schema/`** olarak yeniden adlandırmaktır. Kod ve API taraf...
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
---
## 14. Yapılandırma ve Deterministik Bayrak Yönetimi (`config`)
AmberDB'nin çalıÅma modunu deÄiÅtirmek ve yapılandırma bayraklarını güvenli bir Åekilde yönetmek için `$adb->config()` metodu kullanılır:
```perl
# Toplu veya tekli yapılandırma ataması (Ãnerilen)
$adb->config(
no_write => 1, # Bakım modu: Tüm yazma iÅlemlerini engelle
no_backup => 1, # Tüm tablolar için günlük CSV denetim yedeklerini kapat
use_simple => 1, # İndekssiz doÄrudan yazım modu (İkincil indeksler devre dıÅı bırakılır)
keys_only => 1, # read_all çaÄrılarında sadece ID'leri döndür
ramdisk_size => '1024M', # RAM-Disk (tmpfs/APFS/ImDisk) boyutu (Varsayılan: 512M)
);
# Tekil okuma:
my $no_write = $adb->config('no_write');
# Toplu okuma (Güvenli kopya döner):
my $cfg = $adb->config();
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
```perl
# Bir kaydın kimler tarafından ne zaman deÄiÅtirildiÄini HTML olarak alma
my $gecmis_html = $adb->auth_view("catalog_product", 5001);
print $gecmis_html;
# Ãıktı:
# add 2026-08-14 10:15 admin_maruf
# edit 2026-08-14 11:30 editor_ali
```
### 17.2 Sürekli DeÄiÅiklik AkıÅı (Continuous Recovery Stream - `YYYY-MM-DD.csv`)
AmberDB yapılan her `insert`, `modify` ve `delete` iÅlemini kronolojik zaman-serisi olarak `backup/YYYY/YYYY-MM-DD.csv` dosyasına otomatik olarak ekler (append-only).
Her satır tab ayrılmıŠ(`\t`) olarak Åu sütun yapısında yazılır:
`[Zaman Damgası] \t [Kullanıcı] \t [İÅlem] \t [Tablo] \t [Kayıt ID] \t [PaketlenmiÅ DeÄerler]`
Bu akıÅı devre dıÅı bırakmak için:
* **Tablo Åemasında (Tablo Bazlı):** Åema dosyasına `no_backup => 1` eklenirse sadece o tablo için yedekleme kapatılır.
* **Genel Düzeyde (Tüm Tablolar):** `$adb->config(no_backup => 1);` tanımlanırsa tüm tablolar için yedekleme kapatılır.
### 17.3 Native Veritabanı ArÅivi (`.amberdb` Dump & Restore)
AmberDB, tüm Åemaları (`schema/*.table`, `schema/*.dbase`) ve otoriter veri dosyalarını (`tables/*.db`, `tables/*.del`, `tables/*.aut`, `tables/*.cnt`) SHA-256 doÄrulama özetleriyle birlikte fiziksel dizin yapısıyla birebir örtüÅen sıkı...
TüretilmiÅ indeks dosyaları (`.inx`, `.src`, `.fld`, `.fac`, `.slg`) boyuttan tasarruf etmek için arÅiv içine konmaz; `restore` esnasında Åema kurallarına göre `set_index` ile deterministik olarak sıfırdan üretilir.
```perl
use AmberDB;
use AmberDB::Tools;
my $adb = AmberDB->new(path => { dbase_dir => "./dbstore" });
my $tools = AmberDB::Tools->new($adb);
# 1. Tüm veritabanının tam yedeÄini alma (.amberdb)
my $arsiv = $tools->dump();
# Ãıktı: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb
# 2. Belirli tabloların snapshot yedeÄini alma
$tools->dump(
file => "backup/2026/katalog_yedek.amberdb",
tables => ["catalog_product", "catalog_category"]
);
# 3. YedeÄi geri yükleme ve tüm indeksleri otomatik inÅa etme
$tools->restore(
file => "backup/2026/katalog_yedek.amberdb",
force => 1, # Var olan tabloların üzerine yazma izni
reindex => 1 # İndeksleri sıfırdan üret
);
```
#### CLI Komut Satırı Aracı (`bin/amberdb_backup.pl`)
```bash
# Veritabanını yedekleme
perl bin/amberdb_backup.pl --dump --file backup/2026/tam_yedek.amberdb
# Belirli tabloları yedekleme
perl bin/amberdb_backup.pl --dump --tables products,orders
# YedeÄi güvenli Åekilde geri yükleme
perl bin/amberdb_backup.pl --restore --file backup/2026/tam_yedek.amberdb --force
```
---
## 18. Bakım ve Onarım Araçları (AmberDB::Tools)
Veritabanı indekslerini sıfırdan yeniden oluÅturmak, veri doÄrulaması yapmak veya tabloları optimize etmek için `AmberDB::Tools` kullanılır:
```perl
use AmberDB;
docs/TR.AmberDB_Veritabani_Sistemi.md view on Meta::CPAN
â âââ catalog_product.src â Tam metin arama indeksi
â âââ catalog_product.fac â Facet filtreleme indeksi
â âââ catalog_product.unq â Alan metin sözlüÄü
â âââ catalog_product.slg â Ãift yönlü URL slug haritası ("0:$id", "1:$slug")
â âââ catalog_product.aut â Denetim logu
â âââ catalog_product.del â Silinen kayıtlar
âââ ramdisk/ â RAM-Disk PaylaÅımlı Bellek ve Hızlı Depolama (Linux tmpfs, macOS APFS, Windows ImDisk)
âââ buffer/ â Geçici Disk Buffer / Staging Dosyaları
âââ journal/ â Aktif Transaction Günlükleri (txn_*)
âââ pids/ â Dosya ve Kayıt Kilitleri
âââ backup/ â Günlük CSV Yedekleri
```
---
## 21. GeliÅtirici Tavsiyeleri ve En İyi Pratikler
1. **Toplu Veri GiriÅinde `insert_list` Kullanın:** Yüzlerce kaydı tek tek döngüde `insert_id` ile eklemek yerine tek seferde `insert_list` ile ekleyin; disk I/O ve indeksleme süresi 10 kat hızlanacaktır.
2. **Kritik İŠMantıklarında `transact_start` Kullanın:** Stok düÅme, bakiye güncelleme ve sipariÅ onaylama gibi adımları mutlaka transaction bloÄu içine alın.
3. **Åemalarda Gereksiz Blokları İndekslemeyin:** Yalnızca filtrelenecek alanları `match_block`, aranacak alanları `search_block` olarak tanımlayın.
4. **Sayfalama Dönen DeÄer İmzasını DoÄru KarÅılayın:** `read_all`, `field_fetch` ve `search_table` metotlarında `$limit > 0` verildiÄinde dönen listenin ilk elemanının `$toplam` tamsayısı olduÄunu unutmayın. Asla `my @kayitlar = $a...
lib/AmberDB.pm view on Meta::CPAN
if ($is_async_write) {
$self->ramdisk_mark_dirty( $file_path, $rid, 1 );
}
elsif ( $use_ramdisk == 4 && $is_txn ) {
$self->ramdisk_unmark_dirty( $file_path, $rid );
}
$self->table_close($target_file);
unless ($is_txn) { $self->flock_close( $tableid, $rid ); }
# for index actions and backup
@record = ( $rid, @record );
# Invalidate cached table keys and count in memory
$self->set_cache( $tableid, 'keys', undef );
$self->set_cache( $tableid, 'count', undef );
unless ( $is_async_write || $use_ramdisk == 3 ) {
$self->recs_back( "add", $tableid, \@record );
}
lib/AmberDB.pm view on Meta::CPAN
$self->slug_add( $idx_path, $table_info, $tableid, \@batch );
$self->suggest_add( $idx_path, $table_info, $tableid, \@batch );
unless ($is_async_write) {
if ( $table_info->{log_owner} ) {
foreach my $rec (@batch) {
$self->auth_write( $tableid, $table_path, "add", $rec->[0] );
}
}
unless ( $self->config('no_backup') || $table_info->{no_backup} ) {
$self->recs_back( "add", $tableid, @batch );
}
}
return \%statu;
}
# Replace the DB record with new data.
# ------------------------------------------------
sub update_id {
lib/AmberDB.pm view on Meta::CPAN
$self->table_close($file_path) unless ( $is_async_write || $use_ramdisk == 3 );
$self->table_close($ram_file) if $ram_file;
unless ($is_txn) { $self->flock_close( $tableid, $rid ); }
# Cache invalidate
$self->clear_cache($tableid, $rid);
my @new_rec = ( $rid, @record );
# text backup record.
unless ( $is_async_write || $use_ramdisk == 3 ) {
$self->recs_back( "edit", $tableid, \@new_rec )
or cluck "[DB_TIE] Backup error (edit). $tableid\n";
}
if ( $self->config('use_simple') || ( $table_info && $table_info->{use_simple} ) ) {
$self->_invalidate_child_ramdisk_tables($tableid);
return $rid;
}
lib/AmberDB.pm view on Meta::CPAN
$self->unique_modify( $idx_path, $table_info, \@pairs );
$self->slug_modify( $idx_path, $table_info, $tableid, \@pairs );
if ( $table_info->{log_owner} && !$is_async_write ) {
foreach my $pair (@pairs) {
$self->auth_write( $tableid, $table_path, "edit", $pair->[0] );
}
}
unless ( $is_async_write || $self->config('no_backup') || $table_info->{no_backup} ) {
$self->recs_back( "edit", $tableid, map { $_->[2] } @pairs );
}
return \%statu;
}
# Standard SQL/CRUD aliases
sub modify_id { shift->update_id(@_) }
sub modify_list { shift->update_list(@_) }
lib/AmberDB.pm view on Meta::CPAN
$self->ramdisk_unmark_dirty( $file_path, $rid );
}
$self->table_close($file_path) unless ( $is_async_write || $use_ramdisk == 3 );
$self->table_close($ram_file) if $ram_file;
unless ($is_txn) { $self->flock_close( $tableid, $rid ); }
# Cache invalidate
$self->clear_cache($tableid, $rid);
# Text backup record
unless ( $is_async_write || $use_ramdisk == 3 ) {
$self->recs_back( "del", $tableid, [ $rid, "" ] )
or cluck "[DB_TIE] Backup error (del). $tableid\n";
}
# Move to archive if keep_deleted enabled
if ( $table_info->{keep_deleted} && !$is_async_write ) {
( $self->table_write($del_path)
and $self->recs_put( [ $del_path, $tableid ], [ $rid, $record ] )
and $self->table_close($del_path) )
lib/AmberDB.pm view on Meta::CPAN
}
elsif ( $table_info->{use_facet} ) {
$self->facet_del( $idx_path, $table_info, \@batch );
}
$self->unique_del( $idx_path, $table_info, \@batch );
$self->slug_del( $idx_path, $table_info, $tableid, \@batch );
unless ( $is_async_write || $use_ramdisk == 3 ) {
# Operations: auth, backup
if ( $table_info->{log_owner} ) {
foreach my $rec (@batch) {
$self->auth_write( $tableid, $table_path, "del", $rec->[0] );
}
}
unless ( $self->config('no_backup') || $table_info->{no_backup} ) {
$self->recs_back( "del", $tableid, map { [ $_->[0], "" ] } @batch );
}
}
$self->_invalidate_child_ramdisk_tables($tableid);
return \%statu;
}
# my $ok = $adb->insert_links($tableid, [rid1, lnk1], [rid2, lnk2]);
# Writes alias link bindings into .lnk file.
lib/AmberDB.pm view on Meta::CPAN
if ( $mirror_path && $mirror_path ne $table_path ) {
$self->index_del( [ $mirror_path, $tableid, 1 ], \@keys_to_del );
}
}
}
}
return $del_count;
}
# Writes add|edit|del operation to daily CSV backup audit stream (backup/YYYY/YYYY-MM-DD.csv).
# Exits silently if no_backup is set (globally or in table schema).
# my $ok = $adb->recs_back("add|edit|del", $tableid, @records);
# ------------------------------------------------
sub recs_back {
my ( $self, $action, $tableid, @records ) = @_;
( $action and $tableid and scalar @records ) or return;
# Global config check: disables backup for all tables
return 1 if $self->config('no_backup');
# Table schema check: no_backup => 1 in table schema or volatile Tier 3 RAM-disk
my $table_info = $self->table_info($tableid);
return 1 if $table_info->{no_backup} || ( $table_info->{use_ramdisk} // 0 ) == 3;
my $user = $self->config('user') || 'system';
$tableid =~ s/[:\/\\]/--/g;
my $backup_base = $self->path('backup_dir')
|| ( $self->path('dbase_dir') ? $self->path('dbase_dir') . "/backup" : "backup" );
my $year = $self->year;
my $month = $self->month;
my $day = $self->day;
my $date_iso = "$year-$month-$day";
my $time_str = $self->date_str;
my $backup_file;
if ( $self->config('use_simple') ) {
$backup_file = "$backup_base/$date_iso.csv";
}
else {
my $year_dir = "$backup_base/$year";
$self->make_path($year_dir);
$backup_file = "$year_dir/$date_iso.csv";
}
open my $YAZ, ">>:encoding(UTF-8)", $backup_file
or do {
cluck "[DB_BACKUP] Cannot open backup file $backup_file: $!\n";
return;
};
foreach my $record (@records) {
ref($record) eq "ARRAY" or $record = [$record];
my $rid = $record->[0];
my $bac_val = $self->db_encode( @{$record}[ 1 .. $#$record ] );
print $YAZ "$time_str\t$user\t$action\t$tableid\t$rid\t$bac_val\n";
}
close $YAZ;
lib/AmberDB.pm view on Meta::CPAN
=head1 SUBMODULE ARCHITECTURE & INHERITANCE
C<AmberDB> is built as a unified coordinator that incorporates all functionality from specialized submodules via inheritance (C<use parent>). When you instantiate an C<AmberDB> object (C<$adb>), all methods from the following submodules are directly ...
=over 4
=item * B<L<AmberDB::Date>> - Compact chronological ID getters (C<day_id>, C<second_id>, C<month_id>), date string parsing (C<str2dateid>, C<dateid2str>), range generation (C<day_range>), ISO week numbers (C<dateid2week>), and relative offset calcula...
=item * B<L<AmberDB::Locale>> - Multilingual text processing, locale-aware casing (C<uc>, C<lc>, C<ucfirst>), Unicode Collation (UCA) sorting (C<sort>), ASCII transliteration (C<to_ascii>), number-to-words / cheque conversion (C<num2text>), number fo...
=item * B<L<AmberDB::Tools>> - Enterprise database utilities: backup (C<.amberdb> tar archive with SHA-256 integrity), atomic restore, CSV migration (C<tie2csv>, C<csv2tie>), vacuum compaction, and reindexing.
=back
All submodules (except C<AmberDB::Tools> which takes an C<$adb> handle) can also be instantiated and used independently in standalone scripts.
=head1 TABLE NAMING CONVENTIONS
AmberDB enforces a strict, deterministic lowercase snake_case table naming convention:
=over 4
lib/AmberDB.pm view on Meta::CPAN
=head2 Key Characteristics of Simple Mode
=over 4
=item * B<Arbitrary & Flexible Keys:> The 8-byte ASCII limit and numeric constraints are bypassed. Keys can be emails (C<user@example.com>), UUIDs, long tokens, or Unicode/multilingual strings.
=item * B<Flat Directory Structure:> All tables reside directly under C<dbase_dir> (e.g. C<$dbase_dir/table.db>). No C<table/> or C<schema/> subfolders are required.
=item * B<Rich Nested Structures:> Records can store nested array and hash references (ARRAY/HASH) directly.
=item * B<Continuous Daily Backup Logs:> Text-based continuous daily WAL/CSV logs (C<recs_back>) automatically record all C<add>, C<edit>, and C<del> operations directly into C<$dbase_dir/YYYY-MM-DD.csv> alongside database tables (can be silenced wit...
=item * B<ACID Transactions:> Full multi-table transaction support with atomic rollback (restoring raw records in the C<.db> file).
=item * B<Streaming Queries & Sorting:> Methods like C<read_all>, C<field_fetch>, and C<search_table> operate via direct sequential streaming scans with full support for pagination (C<offset>/C<limit>, with legacy C<start>), C<keys_only>, and in-memo...
=item * B<Zero-Latency RAM-Disk Caching:> Simple mode instances can be initialized directly on RAM-disk mount points (Linux C</dev/shm>, macOS C</Volumes/AmberDB_RAM>, or Windows ImDisk C<R:>) to provide nanosecond-speed transient session and cache s...
=back
=head2 Simple Mode Example
lib/AmberDB.pm view on Meta::CPAN
# 3. Read record (O(1))
my @sess = $adb->read_id('sessions', 'user@example.com');
# 4. Search and filter without indexes
my ($count, @active) = $adb->field_fetch('sessions', 1, 'Active', 0, 10);
# 5. Volatile in-memory simple store
my $ram_db = AmberDB->new(
path => { dbase_dir => "/dev/shm/amberdb_tokens" },
cfg => { use_simple => 1, no_backup => 1 },
);
$ram_db->insert_id('tokens', $token_id, $user_id, time());
=head1 RAM-DISK ACCELERATION
AmberDB features a built-in, transparent physical RAM-disk acceleration engine (Linux C<tmpfs>, macOS C<APFS RAM-Disk> via C<hdiutil>, or Windows C<ImDisk>). It enables high-traffic tables and hot indexes to operate at pure volatile memory speeds whi...
=head2 What is it?
RAM-disk acceleration routes file I/O for database tables directly to an operating system in-memory RAM-disk filesystem (Windows ImDisk C<R:/amberdb_$dbname>, Linux tmpfs C</dev/shm/amberdb_$dbname>, or macOS APFS C</Volumes/amberdb_$dbname>). Unlike...
lib/AmberDB.pm view on Meta::CPAN
my $count = $adb->field_count("catalog_product", 2, 45);
# 1452
When C<$value> is omitted, counts all values indexed for that block:
my $all_counts = $adb->field_count("catalog_product", 2);
# { 45 => 1452, 68 => 21, 712 => 85, ... }
=head2 search_table($table_id, $query, [\%options])
It performs searches matching query terms using the full-text C<.src> index (or a sorted table scan backup method if unindexed). C<search_table> uses the C<AmberDB::Locale> module. It features advanced language normalization according to the selected...
B<IMPORTANT (Return Signature Convention):>
When C<$limit> is passed and C<E<gt> 0> (paginated), C<search_table> returns C<($total_count, @records)> where the first scalar is the total matching count integer. When C<$limit> is omitted or C<0> (unpaginated), it returns C<@records> directly. Unp...
# 1. Unpaginated (returns array of record arrayrefs directly)
my @records = $adb->search_table("catalog_product", "wireless headphones");
my @sorted_records = $adb->search_table("catalog_product", "headphones", { sort => -5 });
# 2. Paginated (first element is total matching count integer)
my ($total_count, @search) = $adb->search_table("catalog_product", "headphones", { offset => 0, limit => 20 });
lib/AmberDB/Base.pm view on Meta::CPAN
# $self->set_datadir("/path/to/dbase")
# ------------------------------------------------
sub set_datadir {
my ( $self, $dbase_dir ) = @_;
$dbase_dir or return;
# declarations (persistent storage only; ramdisk directories are dynamically provisioned on mount)
my @dirs = qw(
dbase_dir table_dir schema_dir backup_dir
buffer_dir journal_dir lock_dir session_dir config_dir
);
foreach my $dir (@dirs) {
$self->{_path}->{$dir} //= "";
}
$self->{_path}->{dbase_dir} = $dbase_dir;
# db_ext tanımlı ve "db" deÄil ise simple moduna al
my $db_ext = $self->config('db_ext');
if ( length($db_ext) && $db_ext ne "db" ) {
$self->config( use_simple => 1 );
}
# do not proceed if simple mode
if ( $self->config('use_simple') ) {
$self->{_path}->{backup_dir} = $dbase_dir;
$self->{_path}->{buffer_dir} = $dbase_dir;
$self->{_path}->{journal_dir} = $dbase_dir;
$self->{_path}->{schema_dir} = $dbase_dir;
$self->{_path}->{table_dir} = $dbase_dir;
$self->{_path}->{lock_dir} = "$dbase_dir/lock";
$self->{_path}->{session_dir} = "$dbase_dir/session";
$self->{_path}->{config_dir} = "$dbase_dir/config";
return 1;
}
$self->{_path}->{journal_dir} = "$dbase_dir/journal";
$self->{_path}->{backup_dir} = "$dbase_dir/backup";
$self->{_path}->{buffer_dir} = "$dbase_dir/buffer";
$self->{_path}->{schema_dir} = "$dbase_dir/schema";
$self->{_path}->{table_dir} = "$dbase_dir/table";
$self->{_path}->{lock_dir} = "$dbase_dir/lock";
$self->{_path}->{session_dir} = "$dbase_dir/session";
$self->{_path}->{config_dir} = "$dbase_dir/config";
# Auto-load connect.pl if present in datadir
my $conn_file = "$dbase_dir/config/connect.pl";
if ( -f $conn_file ) {
lib/AmberDB/Base.pm view on Meta::CPAN
$self->{_connect}->{database} = $cfg->{database};
}
}
unless ( $self->config('test') || $self->config('no_mkdir') || $^C ) {
if ( defined $dbase_dir && $dbase_dir ne "." && $dbase_dir ne "" ) {
for my $dir (
$self->{_path}->{dbase_dir},
$self->{_path}->{table_dir},
$self->{_path}->{schema_dir},
$self->{_path}->{backup_dir},
$self->{_path}->{buffer_dir},
$self->{_path}->{journal_dir},
$self->{_path}->{lock_dir},
$self->{_path}->{session_dir},
) {
if ( defined $dir && length($dir) && !$self->dir_exist($dir) ) {
$self->make_path($dir);
}
}
}
lib/AmberDB/Base.pm view on Meta::CPAN
if ( $key eq 'user' ) {
$self->{_connect}->{username} = $val;
}
}
}
return $self;
}
# my $dbase_dir = $adb->path("dbase_dir"); # getter
# $adb->path({ backup_dir => "$tmpdir/backup" }); # setter
# $adb->path( custom_path => "/opt/data" ); # setter
# ------------------------------------------------
sub path {
my ( $self, @args ) = @_;
# 1. No arguments: return shallow copy of all path mappings
if ( !@args ) {
return { %{ $self->{_path} || {} } };
}
lib/AmberDB/Base/Schema.pm view on Meta::CPAN
our $VERSION = '5.26.6';
my $CREATED = '2026-09-06';
# Whitelist of inheritable group-level configuration keys from .dbase to member tables
our %INHERITABLE_DBASE_KEYS = map { $_ => 1 } qw(
use_ramdisk ramdisk_ttl use_cache cache_ttl
keep_deleted log_owner use_counter
use_section section use_year year
use_language lang
no_transact no_backup table_dir use_junk
use_simple
);
# =====================================================================
# SCHEMA FIELD NORMALIZATION, VALIDATION & TYPE CONVERSION
# =====================================================================
# Normalizes and validates a single field value according to schema block definition.
# Usage:
# my $clean_val = $adb->enc_field($blk_def, $val);
lib/AmberDB/Base/Schema.pm view on Meta::CPAN
if ( ( $schema->{use_ramdisk} // 0 ) == 3 ) {
delete @{$schema}{
qw(
blocks match_block search_block view_block facet_block filter_block
suggest_block suggest_join
slug_block sort_block sort_fields use_facet facet_rules use_junk junk_rules
record_index repeat_start repeat_ids field_rules keep_deleted
)
};
$schema->{use_simple} = 1;
$schema->{no_backup} = 1;
$schema->{no_transact} = 1;
$schema->{ramdisk_ttl} = 300 unless defined $schema->{ramdisk_ttl};
}
# ------------------------------------------------------------------------
# Pipeline Step 2: Simple Mode Stripping
# In simple key-value mode, selectively strip columnar, indexing, relational,
# and caching definitions to enforce lightweight operation.
# ------------------------------------------------------------------------
if ( $schema->{use_simple} && ( $schema->{use_ramdisk} // 0 ) != 3 ) {
lib/AmberDB/Base/Schema.pm view on Meta::CPAN
$table_path =~ s/\//-/g;
my $table_str = "";
# Scalar keys
my @scalar_keys = qw(
name record_index keep_deleted log_owner parent_table
use_menu use_simple force use_ramdisk ramdisk_ttl use_cache use_alias
use_counter use_facet stop_word min_char
use_junk cache_ttl repeat_ids repeat_start
no_transact no_backup table_dir
);
foreach my $key (@scalar_keys) {
next unless exists $tbl->{$key};
next unless defined $tbl->{$key} && $tbl->{$key} ne "";
( my $val = $tbl->{$key} ) =~ s/"/\\"/g;
$table_str .= "\t$key => \"$val\",\n";
}
# Array keys
my @array_keys = qw(
lib/AmberDB/Tools.pm view on Meta::CPAN
AmberDB::Tools - Database maintenance, CLI reindexing, migrations, and bulk conversion toolset
=head1 SYNOPSIS
use AmberDB;
use AmberDB::Tools;
my $adb = AmberDB->new(path => { dbase_dir => "/path/to/dbstore" });
my $tools = AmberDB::Tools->new($adb);
# 1. Create portable .amberdb database backup archive
my $archive = $tools->dump();
# 2. Restore database archive with integrity validation and reindexing
my $result = $tools->restore(file => "backup.amberdb", force => 1);
# 3. Rebuild all indexes for a single table (.inx, .src, .fld, .fac)
$tools->set_index("catalog_product");
# 4. Rebuild only specific index components
$tools->set_search("catalog_product"); # Rebuild full-text search index
$tools->set_fields("catalog_product"); # Rebuild field exact match index
$tools->set_filters("catalog_product"); # Rebuild facet forward filter index
$tools->set_sort("catalog_product"); # Rebuild binary pre-sorted sequences in .inx
lib/AmberDB/Tools.pm view on Meta::CPAN
=head1 DESCRIPTION
C<AmberDB::Tools> provides maintenance, native disaster recovery archiving (C<dump>/C<restore>), batch utility functions for rebuilding indexes, populating full-text search inverted files, compiling forward facet filter dictionaries, generating binar...
Functionality is organized into focused sub-modules:
=over 4
=item * B<AmberDB::Tools::Index>: Index generation routines (C<set_index>, C<set_readall>, C<set_search>, C<set_fields>, C<set_filters>, C<set_rwlnkall>, C<set_sort>, C<index_alltables>, C<check_readall>, C<check_search>).
=item * B<AmberDB::Tools::Maintain>: Maintenance, export/import, and backup routines (C<dump>, C<restore>, C<vacuum>, C<del_table>, C<tie2csv>, C<csv2tie>, C<dir_tables>, C<all_tables>).
=item * B<AmberDB::Tools::Update>: Table format conversions, storage upgrades, and engine updating (C<update_table>, C<update_all>, C<replace_tablename>, C<replace_blockdata>, C<db_simple>, C<convert_tables>, C<update_storage>, C<update_version>).
=back
All operations can also be invoked directly from the terminal via C<amberdb> CLI (e.g. C<amberdb update storage>, C<amberdb update version>, C<amberdb reindex products>).
=head1 CONSTRUCTOR
=head2 new($adb, [%options])
lib/AmberDB/Tools.pm view on Meta::CPAN
=head1 METHODS
=head2 dump([%options])
Creates a compressed, portable C<.amberdb> archive file (gzipped tar archive) containing table and database schemas (C<schema/*.table>, C<schema/*.dbase>), native database data files (C<table/*.db>, C<table/*.del>, C<table/*.aut>, C<table/*.cnt>), an...
Options:
=over 4
=item * C<file>: Custom output file path (defaults to C<backup/YYYY/amberdb_YYYY-MM-DD_time.amberdb>).
=item * C<tables>: Array reference of table IDs to include (defaults to all tables in database).
=item * C<table>: Single table ID to export as a focused snapshot.
=back
my $archive = $tools->dump();
my $archive = $tools->dump(tables => ["catalog_product", "orders_cart"]);
lib/AmberDB/Tools.pm view on Meta::CPAN
=item * C<file>: Path to C<.amberdb> archive file (required).
=item * C<force>: Boolean (default 0). Must be set to 1 to overwrite existing tables in a non-empty database directory.
=item * C<reindex>: Boolean (default 1). Automatically executes C<set_index> for all restored tables.
=item * C<tables>: Array reference of specific table IDs to extract from the archive.
=back
my $res = $tools->restore(file => "backup.amberdb", force => 1);
=head2 set_index($table_id, [@records])
Rebuilds all secondary and primary indexes for C<$table_id> based on its schema definition:
=over 4
=item * Primary key index (C<.inx>) via C<set_readall>
=item * Full-text search inverted indexes (C<.src>) via C<set_search>
lib/AmberDB/Tools.pm view on Meta::CPAN
$tools->set_sort("catalog_product");
=head2 convert_tables()
Scans the entire database directory, identifies all physical tables, and sequentially runs C<set_index> to rebuild and migrate packed binary indexes across the entire system. Returns a status hash reference.
my $status = $tools->convert_tables();
=head2 update_table($table_id, [%options])
Scans an entire table record-by-record, detects legacy formats, creates a timestamped backup, and rewrites the table in native ABR v5 format while rebuilding indexes.
my $res = $tools->update_table("catalog_product", force => 1);
=head2 update_storage([%options])
Migrates storage directories (C<scheme/> to C<schema/>, C<tables/> to C<table/>), converts legacy TSV tables to native ABR v5 format, rebuilds all indexes, and stamps C<config/storage_version.json>.
Options: C<check>, C<force>, C<no_backup>, C<tables>, C<target_dir>.
$tools->update_storage(force => 1);
=head2 update_version([%options])
Queries the MetaCPAN API for newer releases of AmberDB, and optionally invokes C<cpanm> to upgrade the engine.
Options: C<check>, C<cpanm>.
$tools->update_version(check => 1);
lib/AmberDB/Tools/Maintain.pm view on Meta::CPAN
my $adb = $self->{_adb} or return;
$tableid or return;
# file paths
my $table_path = $adb->table_path($tableid);
my $file_path = "${table_path}.$adb->{db_ext}";
my $csv_path = "$table_path.csv";
return unless -e $csv_path;
# backup with timestamp if exists
if ( -e "${table_path}.$adb->{db_ext}" ) {
my $sec_id = $adb->second_id;
rename( "${table_path}.$adb->{db_ext}",
"${table_path}-$sec_id.$adb->{db_ext}" );
unlink("${table_path}.$adb->{db_ext}");
}
# perform conversion operations
my $i = 1;
my @records;
lib/AmberDB/Tools/Maintain.pm view on Meta::CPAN
if ( $adb->config('use_simple') ) {
my $ext = $adb->{db_ext} || "db";
my @files = $adb->dir_files( $dbase_dir, "*.$ext", full_path => 0 );
@all_tables = map { /^([a-z0-9_]+)\.\Q$ext\E$/i ? $1 : () } @files;
}
# 2. Standard Structured Mode: Multi-directory scan (table/ and year directories) for .db files
else {
my $tbl_dir = File::Spec->catdir( $dbase_dir, 'table' );
push @all_tables, $adb->dir_files( $tbl_dir, "*.db", full_path => 0 ) if -d $tbl_dir;
my %seen_dirs = ( "table" => 1, "schema" => 1, "backup" => 1, "lock" => 1, "ramdisk" => 1, "journal" => 1, "session" => 1, "config" => 1 );
if ($year_dir) {
my $yd_path = File::Spec->catdir( $dbase_dir, $year_dir );
push @all_tables, $adb->dir_files( $yd_path, "*.db", full_path => 0 );
$seen_dirs{$year_dir} = 1;
}
# Auto-discover any 4-digit year directories under $dbase_dir (e.g. 2024, 2025, 2026)
if ( -d $dbase_dir ) {
my @year_dirs = $adb->dir_files( $dbase_dir, qr/^\d{4}$/, full_path => 0, files_only => 0 );
foreach my $yd (@year_dirs) {
lib/AmberDB/Tools/Maintain.pm view on Meta::CPAN
# 2. Flush and close open table handles for read consistency
$adb->close_all();
# 3. Determine output file path
my $year = $adb->year;
my $month = $adb->month;
my $day = $adb->day;
my $date_iso = "$year-$month-$day";
my $time_id = $adb->second_id;
my $backup_base = $adb->path('backup_dir')
|| ( $adb->path('dbase_dir') ? $adb->path('dbase_dir') . "/backup" : "backup" );
my $year_dir = "$backup_base/$year";
$adb->make_path($year_dir);
my $outfile = $opts{file} || "$year_dir/amberdb_${date_iso}_${time_id}.amberdb";
# Ensure parent directory for $outfile exists
if ( my ($outdir) = $outfile =~ m{^(.*)[/\\]} ) {
$adb->make_path($outdir);
}
my $tar = Archive::Tar->new();
lib/AmberDB/Tools/Maintain.pm view on Meta::CPAN
cluck "[DB_BACKUP] Failed to write archive $outfile: " . $tar->error() . "\n";
return;
}
$self->{say} .= "Archive successfully written to $outfile (" . ( -s $outfile ) . " bytes)\n";
return wantarray ? ( $outfile, $manifest ) : $outfile;
}
# Restores a .amberdb archive into target database, validates checksums,
# writes data files via native DB_File, and deterministically rebuilds all indexes.
# my $res = $tools->restore( file => 'backup.amberdb', [force => 1], [reindex => 1] );
# ---------------------------------------------------------------------
sub restore {
my ( $self, @args ) = @_;
my $adb = $self->{_adb} or return;
my %opts;
if ( @args == 1 && ref( $args[0] ) eq 'HASH' ) {
%opts = %{ $args[0] };
}
elsif ( @args % 2 == 1 && !ref( $args[0] ) ) {
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
print $out $buf;
}
close $in;
close $out;
return 1;
}
# ---------------------------------------------------------------------
# update_table($tableid, %options)
# Scans an entire table record-by-record, detects any legacy formats (2003-2026),
# creates a timestamped backup with detected dominant version, and rewrites the table
# directly in ABR v1 format preserving original IDs.
# Rebuilds indexes using ($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index($tableid).
# Preserves companion data files ('del', 'aut').
# ---------------------------------------------------------------------
sub update_table {
my ( $self, $tableid, %opts ) = @_;
my $adb = $self->{_adb} or return;
$tableid or return;
my $table_path = $adb->table_path($tableid);
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
# Format modification timestamp: YYYY-MMDD
my @mtime_parts = localtime( ( stat($file_path) )[9] || time() );
my $mtime_str = sprintf( "%04d-%02d%02d", $mtime_parts[5] + 1900, $mtime_parts[4] + 1, $mtime_parts[3] );
# 1. Once mevcut tablodaki kayitlari tara
my %format_counts;
my $total = 0;
my $already_current = 0;
my $legacy_count = 0;
my @decoded_records;
my $recovered_from_backup = 0;
$adb->recs_scan( $file_path, sub {
my ( $k, $v ) = @_;
$total++;
my $fmt = $adb->detect_tsv_format($v);
$format_counts{$fmt}++;
my @fields;
if ( $fmt eq 'v5' ) {
@fields = $adb->db_decode($v);
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
else {
@fields = $adb->tsv_decode( $v, $k );
$legacy_count++;
}
push @decoded_records, [ $k, @fields ];
} );
$adb->table_close($file_path);
# Eger aktif dosya bos ise (ornegin onceki basarisiz bir calismada yedeklenip bos kalmissa):
# Dizin icindeki yedek dosyasini bulup kayitlari oradan kurtar
my $backup_file;
if ( $total == 0 ) {
my ( $pdir ) = $file_path =~ m{^(.*)[/\\]};
$pdir //= ".";
my @cand_backups;
if ( opendir( my $dh, $pdir ) ) {
while ( my $f = readdir($dh) ) {
next if $f eq '.' || $f eq '..';
if ( $f =~ /^\Q$tableid\E-.*?\.\Q$ext\E$/i ) {
push @cand_backups, "$pdir/$f";
}
}
closedir($dh);
}
@cand_backups = sort { ( -s $b ) <=> ( -s $a ) } @cand_backups;
for my $cb (@cand_backups) {
next unless -e $cb && -s $cb;
my @b_records;
my %b_counts;
my $b_legacy = 0;
$adb->recs_scan( $cb, sub {
my ( $k, $v ) = @_;
my $fmt = $adb->detect_tsv_format($v);
$b_counts{$fmt}++;
my @fields = ( $fmt eq 'v5' ) ? $adb->db_decode($v) : $adb->tsv_decode( $v, $k );
$b_legacy++ if $fmt ne 'v5';
push @b_records, [ $k, @fields ];
} );
$adb->table_close($cb);
if (@b_records) {
@decoded_records = @b_records;
%format_counts = %b_counts;
$total = scalar @b_records;
$legacy_count = $b_legacy;
$already_current = $total - $b_legacy;
$recovered_from_backup = 1;
$backup_file = $cb;
last;
}
}
}
# 2. Eslikci veri dosyalarini ('del', 'aut') tara ve formatlarini tespit et
my %companion_info;
my $companion_legacy_total = 0;
for my $cext (qw(del aut)) {
next unless $adb->exist_table( $tableid, $cext );
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
}
if ( $table_info->{match_block} && !-e "$table_path.fld" ) {
$missing_indexes = 1;
}
if ( $table_info->{use_facet} && !-e "$table_path.fac" ) {
$missing_indexes = 1;
}
}
# Eger ana tablo ve tum eslikci dosyalar zaten guncelse, eksik indeks yoksa ve force istenmemisse
if ( $legacy_count == 0 && $companion_legacy_total == 0 && !$opts{force} && !$recovered_from_backup && !$missing_indexes ) {
$self->{say} .= "Table '$tableid' is already up to date in ABR v1 format ($already_current records).\n";
my $res = {
status => 'already_current',
table => $tableid,
total => $total,
updated => 0,
already_current => $already_current,
};
for my $cext (qw(del aut)) {
if ( my $ci = $companion_info{$cext} ) {
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
my $dom_ver = 'v4';
my $max_c = -1;
for my $v (qw(v4 v3 v2 v1)) {
if ( ( $format_counts{$v} // 0 ) > $max_c ) {
$max_c = $format_counts{$v} // 0;
$dom_ver = $v;
}
}
# 4. Ana tablo donusumu (gerekliyse veya force ise)
if ( $legacy_count > 0 || $recovered_from_backup || $opts{force} ) {
# Eger yedekten kurtarilmadiysa, aktif dosyayi nihai yedek adiyla yedekle
if ( !$recovered_from_backup ) {
my $backup_base = "$table_path-$dom_ver-$mtime_str";
$backup_file = "$backup_base.$ext";
my $counter = 1;
while ( -e $backup_file ) {
$backup_file = "$backup_base-$counter.$ext";
$counter++;
}
rename( $file_path, $backup_file ) or do {
cluck "[DB_TOOL] Could not backup $file_path to $backup_file: $!\n";
return { status => 'error', table => $tableid, error => "Backup failed: $!" };
};
}
# Yeni dosyayi sifir temiz yazma modunda ac
$adb->table_close($file_path);
$adb->clear_cache($tableid);
my $new_db = $adb->table_write($file_path);
unless ($new_db) {
return { status => 'error', table => $tableid, error => "Could not open $file_path for writing" };
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
$adb->table_close($file_path);
# Eski turetilen indeks dosyalarini temizle
for my $iext (qw(inx src fld fac slg ajw ajn)) {
my $idx_f = "$table_path.$iext";
unlink $idx_f if -e $idx_f;
}
$adb->clear_cache($tableid);
}
else {
$backup_file //= "None (already ABR v1)";
$self->{say} .= " - Main table is already in ABR v1 format ($already_current records).\n";
}
# 5. Eslikci veri dosyalarini ('del', 'aut') donustur (indeks insasindan once calisir)
my %companion_stats;
for my $cext (qw(del aut)) {
my $ci = $companion_info{$cext};
next unless $ci;
my $cfile = $ci->{file};
if ( $ci->{legacy} > 0 || $opts{force} ) {
if ( $ci->{total} > 0 ) {
$self->{say} .= " - Migrating companion file ($cext: $ci->{total} records)...\n";
my $c_backup_base = "$table_path-$dom_ver-$mtime_str.$cext";
my $c_backup = $c_backup_base;
my $cnt = 1;
while ( -e $c_backup ) {
$c_backup = "$table_path-$dom_ver-$mtime_str-$cnt.$cext";
$cnt++;
}
rename( $cfile, $c_backup ) or next;
my $new_cdb = $adb->table_write($cfile);
if ($new_cdb) {
for my $crec ( @{ $ci->{decoded} } ) {
my ( $ck, @cfields ) = @$crec;
my $cv_new = $adb->db_encode(@cfields);
$new_cdb->put( $adb->utf_encode("$ck"), $adb->utf_encode("$cv_new") );
}
$adb->table_close($cfile);
$companion_stats{"${cext}_migrated"} = $ci->{total};
$companion_stats{"${cext}_backup"} = $c_backup;
}
else {
rename( $c_backup, $cfile );
}
}
else {
$companion_stats{"${cext}_status"} = "0 records (empty)";
}
}
else {
$companion_stats{"${cext}_status"} = "already ABR v1 ($ci->{total} records)";
}
}
# 6. Indeksleri insa et (gerekliyse veya eksikse)
my $is_simple = $adb->config('use_simple') || ( $table_info && $table_info->{use_simple} ) || ( $table_info && $table_info->{id_type} && $table_info->{id_type} eq 'ascii' );
if ( $is_simple ) {
$self->{say} .= " - Table '$tableid' is in simple mode, skipping index generation.\n";
}
elsif ( $legacy_count > 0 || $recovered_from_backup || $opts{force} ) {
$self->{say} .= " - Rebuilding indexes...\n";
eval {
($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index( $tableid, @decoded_records );
1;
} or do {
warn " [ERROR in set_index]: $@\n";
};
}
elsif ($missing_indexes) {
$self->{say} .= " - Rebuilding missing indexes...\n";
eval {
($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index( $tableid, @decoded_records );
1;
} or do {
warn " [ERROR in set_index]: $@\n";
};
}
# 7. Eslikci sozluk ve sayac (.unq, .cnt) dosyalarini koru ve yedekle
my $has_unq = $adb->exist_table($tableid, 'unq');
my $unq_backup;
if ($has_unq) {
my $unq_file = "$table_path.unq";
my $u_backup_base = "$table_path-$dom_ver-$mtime_str.unq";
$unq_backup = $u_backup_base;
my $cnt = 1;
while ( -e $unq_backup ) {
$unq_backup = "$table_path-$dom_ver-$mtime_str-$cnt.unq";
$cnt++;
}
require File::Copy;
File::Copy::copy( $unq_file, $unq_backup );
}
my $has_cnt = $adb->exist_table($tableid, 'cnt');
my $cnt_backup;
if ($has_cnt) {
my $cnt_file = "$table_path.cnt";
my $c_backup_base = "$table_path-$dom_ver-$mtime_str.cnt";
$cnt_backup = $c_backup_base;
my $cnt = 1;
while ( -e $cnt_backup ) {
$cnt_backup = "$table_path-$dom_ver-$mtime_str-$cnt.cnt";
$cnt++;
}
require File::Copy;
File::Copy::copy( $cnt_file, $cnt_backup );
}
$self->{say} .= "Table '$tableid' updated to ABR v1: $total records migrated ($legacy_count converted, backup: $backup_file).\n";
return {
status => ( ( $legacy_count > 0 || $companion_legacy_total > 0 || $recovered_from_backup || $opts{force} ) ? 'updated' : 'already_current' ),
table => $tableid,
total => $total,
updated => $legacy_count,
already_current => $already_current,
dominant_format => $dom_ver,
backup_file => $backup_file,
del_migrated => $companion_stats{del_migrated} // 0,
del_backup => $companion_stats{del_backup},
del_status => $companion_stats{del_status},
aut_migrated => $companion_stats{aut_migrated} // 0,
aut_backup => $companion_stats{aut_backup},
aut_status => $companion_stats{aut_status},
has_unq => $has_unq ? 1 : undef,
unq_backup => $unq_backup,
has_cnt => $has_cnt ? 1 : undef,
cnt_backup => $cnt_backup,
};
}
# ---------------------------------------------------------------------
# update_all(%options)
# Iterates through all discovered tables and runs update_table on each.
# ---------------------------------------------------------------------
sub update_all {
my ( $self, %opts ) = @_;
my $adb = $self->{_adb} or return;
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
$converted{$tableid} = 1;
$self->{say} .= " -> Rebuilt packed binary indexes (.inx, .fld, .src) for $tableid\n";
}
}
return \%converted;
}
# Creates a compressed, portable .amberdb archive containing table schemas,
# native data files (.db, .del, .aut, .cnt), and integrity manifest.
# my $archive = $tools->dump( [file => 'backup.amberdb'], [tables => ['t1', 't2']] );
# ---------------------------------------------------------------------
# ---------------------------------------------------------------------
# update_version(%opts)
# Checks MetaCPAN API for latest AmberDB release and optionally installs via cpanm.
# ---------------------------------------------------------------------
sub update_version {
my ( $self, %opts ) = @_;
my $opt_check = $opts{check} // 0;
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
$target_dir = $cwd;
}
else {
$target_dir = $cwd;
}
}
$target_dir = eval { Cwd::abs_path($target_dir) } // $target_dir;
my $opt_force = $opts{force} // 0;
my $opt_check = $opts{check} // 0;
my $opt_no_backup = $opts{no_backup} // $opts{'no-backup'} // 0;
my $opt_tables = $opts{tables} // $opts{table} // '';
my $opt_manifest = $opts{manifest} // '';
print "=================================================================\n";
print " AmberDB Storage, Directory & Compatibility Migration Engine \n";
print "=================================================================\n";
print "Database Directory : $target_dir\n";
my $config_dir = File::Spec->catdir( $target_dir, "config" );
my $ver_file = File::Spec->catfile( $config_dir, "storage_version.json" );
lib/AmberDB/Tools/Update.pm view on Meta::CPAN
}
if ( $v_curr < version->parse('5.25.0') || $opt_force ) {
print " - [v5.25.0] Directory migration: Rename tables/ -> table/\n";
print " - [v5.25.0] Format migration: Convert legacy records to ABR v5 binary pack\n";
print " - [v5.25.0] Index migration: Rebuild all secondary indexes (.inx, .fld, .unq, .fac, .slg, .srt)\n";
}
print "=================================================================\n";
return { status => 'check', current_version => $current_storage_ver, target_version => $target_storage_ver };
}
# 3. Pre-migration Safety Snapshot (unless --no-backup)
if ( !$opt_no_backup ) {
print "Taking pre-migration safety snapshot...\n";
my $backup_file = File::Spec->catfile( $target_dir, "backup_pre_storage_update_" . time() . ".amberdb" );
eval {
require AmberDB;
my $snap_adb = AmberDB->new( path => { dbase_dir => $target_dir } );
my $maintainer = $self->can('dump') ? $self : do {
require AmberDB::Tools::Maintain;
AmberDB::Tools::Maintain->new($snap_adb);
};
$maintainer->dump( file => $backup_file );
};
if ( -e $backup_file && -s $backup_file ) {
print " [+] Safety snapshot created: $backup_file\n";
}
else {
print " [INFO] Snapshot skipped or empty database.\n";
}
}
# 4. Stage v5.21.0 Migration: Rename scheme -> schema
if ( $v_curr < version->parse('5.21.0') ) {
print "\n>>> [Stage v5.21.0 Migration] Scheme to Schema Directory Renaming...\n";
$self->_inline_migrate_5_21_0($target_dir);
migrations/convert_allkeys.pl view on Meta::CPAN
my @files = sort readdir($dh);
closedir($dh);
my %tables;
my %side_files = ( del => 0, aut => 0, cnt => 0 );
foreach my $file ( sort @files ) {
next if $file =~ /^\./; # skip . and .. and hidden files
$file =~ s|\Q$tables_dir\E/?||i;
next if $file =~ /^\_/; # skip temporary/hidden files
next if $file =~ /\s/; # skip backup filenames with spaces
if ( $file =~ /^([a-z0-9_]+)\.(db|inx)$/i ) {
$tables{$1} = 1;
}
elsif ( $file =~ /\.del$/i ) {
$side_files{del}++;
}
elsif ( $file =~ /\.aut$/i ) {
$side_files{aut}++;
}
migrations/update_tables.pl view on Meta::CPAN
#!/usr/bin/perl
# bin/update_tables.pl - Automated Table Migration Utility for AmberDB
# Migrates historical tables (2003-2026 formats) to the latest ABR v1 binary format.
# Automatically creates timestamped backups with detected version tag:
# catalog_product.db -> catalog_product-v3-2018-0315.db
use 5.016;
use strict;
use warnings;
use Getopt::Long qw(GetOptions);
use FindBin;
use lib "$FindBin::Bin/../lib";
use AmberDB;
migrations/update_tables.pl view on Meta::CPAN
my $err = $res->{error} // 'Unknown error';
print " [ERROR] FAILED! ($err)\n";
next;
}
$total_processed++;
if ($res->{status} eq 'already_current') {
print " [OK] ALREADY ABR v1 (" . ($res->{already_current} // 0) . " records)\n";
if ($res->{del_migrated}) {
print " - Archive (.del) : $res->{del_migrated} records migrated (Backup: $res->{del_backup})\n";
}
elsif (defined $res->{del_status}) {
print " - Archive (.del) : $res->{del_status}\n";
}
if ($res->{aut_migrated}) {
print " - Audit Log (.aut) : $res->{aut_migrated} entries migrated (Backup: $res->{aut_backup})\n";
}
elsif (defined $res->{aut_status}) {
print " - Audit Log (.aut) : $res->{aut_status}\n";
}
$total_current++;
}
elsif ($res->{status} eq 'updated') {
print " [OK] MIGRATED!\n";
print " - Records Migrated : $res->{total} (Legacy: $res->{updated}, ABR: $res->{already_current})\n";
print " - Dominant Format : $res->{dominant_format}\n";
print " - Backup Created : $res->{backup_file}\n";
if ($res->{del_migrated}) {
print " - Archive (.del) : $res->{del_migrated} records migrated (Backup: $res->{del_backup})\n";
}
elsif (defined $res->{del_status}) {
print " - Archive (.del) : $res->{del_status}\n";
}
if ($res->{aut_migrated}) {
print " - Audit Log (.aut) : $res->{aut_migrated} entries migrated (Backup: $res->{aut_backup})\n";
}
elsif (defined $res->{aut_status}) {
print " - Audit Log (.aut) : $res->{aut_status}\n";
}
if ($res->{has_unq}) {
print " - Unique (.unq) : Preserved (Backup: $res->{unq_backup})\n";
}
if ($res->{has_cnt}) {
print " - Counter (.cnt) : Preserved (Backup: $res->{cnt_backup})\n";
}
$total_updated++;
$total_records += $res->{total};
}
else {
print "$res->{status}\n";
}
}
print "=================================================================\n";
t/amberdb_backup.t view on Meta::CPAN
use AmberDB::Tools;
my $tmpdir = tempdir( CLEANUP => 1 );
$tmpdir =~ s{\\}{/}g;
my $adb = AmberDB->new(
path => { dbase_dir => $tmpdir },
cfg => { user => "admin", language => "tr" },
);
ok( defined $adb, "AmberDB instance created for backup tests" );
# =========================================================================
# SUBTEST 1: recs_back Daily WAL CSV Stream (backup/YYYY/YYYY-MM-DD.csv)
# =========================================================================
subtest "1. Continuous Recovery Stream (recs_back -> YYYY-MM-DD.csv)" => sub {
plan tests => 6;
# Create a test table
my $schema = {
name => 'audit_items',
record_index => 1,
search_block => [ 1 ],
log_owner => 1,
t/amberdb_backup.t view on Meta::CPAN
# Insert a record
my $rid = $adb->insert_id( 'audit_items', undef, 'Keyboard', 450 );
ok( $rid > 0, "Record inserted with ID $rid" );
# Edit record
$adb->modify_id( 'audit_items', $rid, 'Mechanical Keyboard', 550 );
# Delete record
$adb->delete_id( 'audit_items', $rid );
# Verify backup directory structure
my $year = $adb->{date}->{year};
my $date_iso = "$adb->{date}->{year}-$adb->{date}->{month}-$adb->{date}->{day}";
my $csv_file = "$tmpdir/backup/$year/$date_iso.csv";
ok( -d "$tmpdir/backup/$year", "Year directory backup/$year created" );
ok( -e $csv_file, "Daily CSV stream file $csv_file created" );
# Read CSV lines
open my $fh, "<:encoding(UTF-8)", $csv_file or die "Cannot open $csv_file: $!";
my @lines = <$fh>;
close $fh;
ok( scalar(@lines) >= 3, "CSV contains at least 3 log entries (add, edit, del)" );
# Check line format: timestamp \t user \t action \t table \t rid \t values
t/amberdb_backup.t view on Meta::CPAN
$adb->insert_id( 'catalog_product', undef, 'Desk Lamp', 'Furniture', 350 );
$adb->insert_id( 'catalog_product', undef, 'Office Chair', 'Furniture', 1200 );
# Create a unique/dictionary index (.unq) for block 2
$adb->insert_strs( 'catalog_product', 2, [ 10, 'Electronics' ], [ 20, 'Furniture' ] );
$adb->insert_id( 'orders', undef, 'Alice', 15350 );
$adb->insert_id( 'orders', undef, 'Bob', 1200 );
# Run full dump
my $dump_file = "$tmpdir/backup/test_dump.amberdb";
my ($outfile, $manifest) = $tools->dump( file => $dump_file );
ok( defined $outfile, "dump() returned output path" );
is( $outfile, $dump_file, "dump() created requested archive path" );
ok( -e $dump_file, "Archive file exists on disk" );
ok( -s $dump_file > 0, "Archive file is non-empty" );
# Validate returned in-memory manifest structure
ok( defined $manifest, "dump() returned manifest hashref" );
is( $manifest->{format}, 'AmberDB Archive', "Manifest format is 'AmberDB Archive'" );
t/amberdb_backup.t view on Meta::CPAN
ok( exists $manifest->{tables}->{catalog_product}, "Manifest contains catalog_product table metadata" );
is( $manifest->{tables}->{catalog_product}->{records}, 3, "Manifest records 3 records for catalog_product" );
};
# =========================================================================
# SUBTEST 3: Archive Package Inspection (.tar Extraction & Index Exclusion)
# =========================================================================
subtest "3. Archive Package Inspection & Integrity" => sub {
plan tests => 8;
my $dump_file = "$tmpdir/backup/test_dump.amberdb";
my $tar = Archive::Tar->new();
ok( $tar->read($dump_file), "Archive is a valid tar archive" );
my @files = $tar->list_files();
ok( grep { $_ eq 'manifest.json' } @files, "Archive contains manifest.json" );
ok( grep { $_ eq 'schema/catalog.dbase' } @files, "Archive contains schema/catalog.dbase" );
ok( grep { $_ eq 'schema/catalog_product.table' } @files, "Archive contains schema/catalog_product.table" );
ok( grep { $_ eq 'table/catalog_product.dumpdb' } @files, "Archive contains table/catalog_product.dumpdb" );
ok( grep { $_ eq 'table/orders.dumpdb' } @files, "Archive contains table/orders.dumpdb" );
t/amberdb_backup.t view on Meta::CPAN
my $sha = $manifest->{tables}->{catalog_product}->{sha256}->{'table/catalog_product.dumpdb'};
ok( defined $sha && length($sha) == 64, "Manifest contains valid 64-character SHA-256 hash" );
};
# =========================================================================
# SUBTEST 4: Tools->restore() and Deterministic Index Rebuilding
# =========================================================================
subtest "4. Database Restore and Index Reconstruction" => sub {
plan tests => 17;
my $dump_file = "$tmpdir/backup/test_dump.amberdb";
# 4.1 Safety check: Restore into non-empty database without force should fail
my $tools = AmberDB::Tools->new($adb);
my $blocked_res = $tools->restore( file => $dump_file, force => 0 );
ok( !defined $blocked_res, "restore() without force on non-empty database returns undef" );
# 4.2 Create a completely clean staging database directory
my $stagedir = tempdir( CLEANUP => 1 );
$stagedir =~ s{\\}{/}g;
t/amberdb_encapsulation.t view on Meta::CPAN
plan tests => 6;
# Getter
is( $adb->path('dbase_dir'), $tmpdir, "path('dbase_dir') returns expected directory" );
# Setter via key-value
$adb->path( custom_path => "/opt/data" );
is( $adb->path('custom_path'), "/opt/data", "path(custom_path => ...) sets path" );
# Setter via hashref
$adb->path({ backup_dir => "$tmpdir/backup" });
is( $adb->path('backup_dir'), "$tmpdir/backup", "path({ backup_dir => ... }) sets path" );
# Bulk copy
my $paths_copy = $adb->path();
ok( ref($paths_copy) eq 'HASH', "path() returns HASH ref" );
is( $paths_copy->{dbase_dir}, $tmpdir, "path copy contains dbase_dir" );
# Mutation protection
$paths_copy->{dbase_dir} = "/compromised/path";
is( $adb->path('dbase_dir'), $tmpdir, "Internal path is NOT affected by mutating copy" );
};
t/amberdb_simple_mode.t view on Meta::CPAN
# 2. Modify record (edit action)
$adb->modify_id( 'sessions', $sid, 'Closed', '192.168.1.50' );
# 3. Delete record (del action)
$adb->delete_id( 'sessions', $sid );
my $date_iso = "$adb->{date}->{year}-$adb->{date}->{month}-$adb->{date}->{day}";
my $csv_file = "$db_dir/$date_iso.csv";
ok( -e $csv_file, "Daily CSV backup file $csv_file created directly in same dbase_dir in simple mode" );
ok( !-d "$db_dir/backup", "No separate backup directory created in simple mode" );
open my $fh, "<:encoding(UTF-8)", $csv_file or die "Cannot open $csv_file: $!";
my @lines = <$fh>;
close $fh;
is( scalar(@lines), 3, "CSV contains exactly 3 entries (add, edit, del) in simple mode" );
# Line 1: add
chomp $lines[0];
my @cols_add = split /\t/, $lines[0];
t/amberdb_simple_mode.t view on Meta::CPAN
my @cols_edit = split /\t/, $lines[1];
is( $cols_edit[2], 'edit', "Col 2 is action 'edit'" );
is( $cols_edit[4], $sid, "Col 4 is custom string ID" );
# Line 3: del
chomp $lines[2];
my @cols_del = split /\t/, $lines[2];
is( $cols_del[2], 'del', "Col 2 is action 'del'" );
is( $cols_del[4], $sid, "Col 4 is custom string ID" );
# 4. Verify no_backup flag disables logging
my $db_dir_noback = "$tmp_dir/simple_db8_noback";
mkdir($db_dir_noback);
my $adb_noback = AmberDB->new(
path => { dbase_dir => $db_dir_noback },
cfg => { use_simple => 1, no_backup => 1 },
);
$adb_noback->insert_id( 'tbl', 'id1', 'val1' );
my $noback_csv = "$db_dir_noback/$date_iso.csv";
ok( !-e $noback_csv, "No backup CSV created when no_backup => 1" );
};
# ============================================================
# 9. Simple Mode Key Sanitization and Reference Rejection
# ============================================================
subtest '9. Simple Mode Key Sanitization and Reference Rejection' => sub {
my $db_dir = "$tmp_dir/simple_db9";
mkdir($db_dir);
my $adb = $tools->db_simple($db_dir);
t/amberdb_update_table.t view on Meta::CPAN
$dbh->put("2", "Product 2005\tred\\Tblue");
$dbh->put("3", "Product 2021<TAB0>opt1<TAB1>opt2");
$dbh->put("4", "Product 2026\tARRAY:t1|t2\tHASH:k=v");
$adb->table_close($table_file);
# Run update_table
my $res = $tools->update_table($tbl);
is( $res->{status}, 'updated', 'update_table returned status updated' );
is( $res->{total}, 4, 'Total records processed is 4' );
is( $res->{updated}, 4, '4 legacy records were converted' );
ok( -e $res->{backup_file}, "Backup file was created: $res->{backup_file}" );
like( $res->{backup_file}, qr/catalog_products-v\d-\d{4}-\d{4}\.db/, 'Backup file matches -v<ver>-<date>.db naming pattern' );
# Verify table is now 100% ABR v1
$adb->table_read($table_file);
my %migrated;
$adb->recs_scan($table_file, sub {
my ($k, $v) = @_;
$migrated{$k} = $v;
});
$adb->table_close($table_file);
t/amberdb_update_table.t view on Meta::CPAN
# Write .del archive in 2005 \T format
$adb->table_write($del_file);
my $ddbh = $adb->{_db}->{$del_file};
$ddbh->put("999", "Order Cancelled\titem1\\Titem2\t0");
$adb->table_close($del_file);
# Run update_table
my $res = $tools->update_table($tbl);
is( $res->{status}, 'updated', 'Orders table updated' );
is( $res->{del_migrated}, 1, '1 record in .del archive migrated' );
ok( -e $res->{del_backup}, "Backup file for .del created: $res->{del_backup}" );
# Verify .del file is now ABR v1
$adb->table_read($del_file);
my %del_migrated;
$adb->recs_scan($del_file, sub {
my ($k, $v) = @_;
$del_migrated{$k} = $v;
});
$adb->table_close($del_file);
t/amberdb_update_table.t view on Meta::CPAN
is( $adb->exist_table($tbl, 'aut'), 1, 'exist_table detects .aut' );
is( $adb->exist_table($tbl, 'cnt'), 1, 'exist_table detects .cnt' );
is( $adb->exist_table($tbl, 'slg'), 0, 'exist_table returns 0 for non-existent .slg' );
# Run update_table
my $res = $tools->update_table($tbl);
is( $res->{status}, 'updated', 'Products table updated successfully' );
# Verify .unq was PRESERVED and NOT unlinked
is( $res->{has_unq}, 1, 'Report acknowledges .unq was found' );
ok( -e $res->{unq_backup}, "Backup copy of .unq created: $res->{unq_backup}" );
is( $adb->exist_table($tbl, 'unq'), 1, 'Live .unq file is preserved and still exists!' );
# Verify data inside live .unq was NOT deleted or corrupted
$adb->table_read($unq_file);
my $unq_val = $adb->recs_get($unq_file, "title:s:Wireless Keyboard");
$adb->table_close($unq_file);
is( $unq_val->{"title:s:Wireless Keyboard"}, 42, '.unq dictionary mapping retained completely' );
# Verify .aut was migrated and backed up
is( $res->{aut_migrated}, 1, '.aut audit entry migrated' );
ok( -e $res->{aut_backup}, "Backup of .aut created: $res->{aut_backup}" );
# Verify .cnt was preserved and backed up
is( $res->{has_cnt}, 1, '.cnt counter preserved' );
};
done_testing();