AmberDB

 view release on metacpan or  search on metacpan

Changes  view on Meta::CPAN

            - Getter: $adb->config('simple') is retained as a transparent fallback and resolves to $self->{_cfg}->{use_simple}.
            - Setter: $adb->config(simple => $val) automatically maps to $self->{_cfg}->{use_simple}.
            - Constructor: AmberDB->new(cfg => { simple => 1 }) automatically normalizes input to { use_simple => 1 }.
          * Updated test suite (t/amberdb_simple_mode.t) to assert single-key storage in _cfg and verify two-way getter/setter interoperability between 'simple' and 'use_simple'.

        - [NAMESPACE CLEANUP & REMOVAL OF AMBIGUOUS METHOD ALIASES]
          * Removed legacy typeglob aliases ('str' and 'short' in AmberDB::Date, 'by_code' and 'all' in AmberDB::Locale::Currency, and 'reindex' in AmberDB::Tools::Index) to prevent ambiguous method pollution on inherited AmberDB instances and elimin...

5.26.5  2026-10-10
        - [CROSS-PLATFORM ARCHIVE RESTORE & DUMP SYSTEM]
          * Refactored backup archive dump and restore (AmberDB::Tools::Maintain) to support cross-platform portable exports (.dumpdb, .dumpdel, .dumpaut, .dumpcnt) utilizing TSV and Base64 serialization, eliminating Berkeley DB endianness/binary inc...
          * Rewrote restore() and dump() to strictly use encapsulated AmberDB table_write(), table_read(), recs_scan(), and table_close() methods, fully eliminating raw DB_File and tie/untie usage.
          * Enhanced restore() argument parsing to accept single-string file path ($tools->restore('backup.amberdb')), hashref, or key-value pairs.
          * Added automatic schema cache invalidation (delete $adb->{_table}->{$tid}) upon restoring tables to ensure subsequent schema introspection reloads cleanly.
          * Added resilient error handling with eval surrounding index rebuilds during restore.

        - [TABLE COUNT & LOCK PATH ROBUSTNESS]
          * Added definedness guard for raw index key string in AmberDB::table_count.
          * Standardized lock directory resolution via $self->path("lock_dir") in flock_open.
          * Extended $adb->path() to support setter operations ($adb->path({ backup_dir => '...' }) and $adb->path(key => 'val')).

5.26.4  2026-10-01
        - [IMPROVEMENTS] 
          * The reindex vacuum bug has been resolved.
          * Some incorrect statements in "Changes" have been corrected.

5.26.3  2026-09-29
        - [AUTOCOMPLETE & INSTANT SEARCH / AJAX QUERY SUGGESTION ENGINE (.ajw, .ajn)]
          * Implemented high-performance dual-index query suggestion and autocomplete engine designed for Ajax instant-search bars and typeahead UI widgets.
          * Introduced '.ajw' (Word Prefix Index): maps 2 to 12-character token prefixes to top candidate words with occurrence frequency scores (top 10 matches per prefix), supporting both native Unicode (Turkish/international alphabet) and ASCII-fo...

Changes  view on Meta::CPAN

        - [RELATIONAL PARENT/CHILD TABLES & MATCH INDEXING]
          * Enforced transparent child table RAM-disk cache invalidation (_invalidate_child_ramdisk_tables) when parent_table receives writes or deletes.
          * Validated match_block index construction and field_fetch O(1) lookups for child tables linked by parent_table (e.g., member_google).

        - [ENCODER & BINARY BUFFER OPTIMIZATIONS]
          * Optimized binary crop routines (t/amberdb_bin_crop.t) and range filtering algorithms in AmberDB::Base::Encoder and AmberDB::Array.

        - [TOOLS REFACTOR & SUB-MODULE DECOMPOSITION (AmberDB::Tools)]
          * Decomposed monolithic AmberDB::Tools (~2350 lines) into modular sub-modules under lib/AmberDB/Tools/:
            - AmberDB::Tools::Index: Primary keys, full-text inverted search, columnar facets, match blocks, sort matrices, and URL slug indexing (set_index, set_readall, set_search, set_fields, set_filters, set_rwlnkall, set_sort, index_alltables, c...
            - AmberDB::Tools::Maintain: Database maintenance, backup dump/restore, compacting, table deletion, and TSV/CSV conversion (dump, restore, vacuum, del_table, tie2csv, csv2tie, dir_tables, all_tables, _copy_file).
            - AmberDB::Tools::Update: Table upgrades, schema migrations, and centralized update routines (update_table, update_all, replace_tablename, replace_blockdata, db_simple, convert_tables, update_storage, update_version).
          * Preserved AmberDB::Tools as the unified facade wrapper inheriting from all sub-modules (use parent qw(...)) with 100% backward compatibility.
          * Enforced strict zero-POD rule across all sub-modules; all user-facing documentation is consolidated in AmberDB::Tools.

        - [CENTRALIZED CLI STORAGE & VERSION UPDATES (amberdb_cli.pl)]
          * Integrated 'amberdb update storage' and 'amberdb update version' into bin/amberdb_cli.pl (and amberdb/amberdb.bat wrappers).
          * Added CLI option parsing for --check, --force, --all, --no-backup, --tables, and --cpanm flags.
          * Fully eliminated legacy bin/amberdb_setup.pl; all maintenance and provisioning tasks are routed through the main amberdb CLI.
          * Registered bin/amberdb and bin/amberdb_cli.pl in Makefile.PL EXE_FILES.

        - [RAM-DISK ZERO-FALLBACK & SHARED MEMORY ARCHITECTURE]
          * Re-engineered RAM-disk mounting lifecycle using platform table mapping (%RM_DIR: Windows R:, Linux /dev/shm, macOS /Volumes).
          * Fixed Windows ImDisk detection in AmberDB::Base::Ramdisk by appending a trailing slash ($rm_base/) in -d tests to resolve MSYS2/Windows drive root stat quirks.
          * Enforced strict Zero-Fallback architecture: when unmounted or disconnected, ramdisk_dir, table_rdir, schema_rdir, and config_rdir are normalized strictly to "" (empty string) in both $cfg and {_path}.
          * Completely removed shmem_dir and any fallback creation of dbstore/ramdisk directories on physical disk when unmounted; tables and routines never assume a RAM-disk is present.
          * Removed table_rdir, schema_rdir, and config_rdir from default persistent @dirs initialization array in AmberDB::Base::set_datadir.
          * Implemented pure RAM-disk Shared Memory API: get_shmem(), set_shmem(), del_shmem() with strict zero-fallback to disk when unmounted.

Changes  view on Meta::CPAN

          * Added candidate pruning for small survivor sets (<= 250 records) directly on the raw binary slice.
          * Full lifecycle management for $blk:vals: automatically maintains and prunes values across sort_add, sort_modify, and sort_del.

        - [FIELD QUERY ENHANCEMENTS]
          * field_fetch now accepts named block arguments via resolve_block_idx().
          * Fixed raw numeric value recovery for fallback queries on normalized .inx sort keys.

        - [DATE ENGINE OPTIMIZATIONS]
          * Added memoization caching to AmberDB::Date with rollover protection.
          * Added reset_date() methods.
          * Replaced direct hash access with fast accessor methods across dump, vacuum, and backup routines.

        - [DIRECTORY ARCHITECTURE & LOCK PATH STANDARDIZATION]
          * Standardized lock_dir and session_dir as single-key directories in AmberDB::Base::Ramdisk and AmberDB::Base.
          * Removed obsolete fallback keys (lock_rdir) and legacy naming aliases (conf_dir, cfg_dir).
          * In RAM-disk environments, lock_dir and session_dir are dynamically mounted under $ramdisk_dir/lock and $ramdisk_dir/session with 0755 permissions; when unmounted, they reside cleanly under $dbase_dir/lock and $dbase_dir/session.
          * Aligned database lock directory structures with inter-process state locking specifications.

        - [SCHEMA LOADING RESILIENCE]
          * Wrapped 'do $target_path' inside eval { do $target_path } in AmberDB::Base::Schema (table_info() and dbase_info()).
          * Prevents fatal process/worker termination when user .table or .dbase schema files contain syntax errors under global $SIG{__DIE__} handlers.

Changes  view on Meta::CPAN

          * Automated Timestamped Backups: Automatically backs up migrating tables as <table_name>-v<detected_ver>-<YYYY-MMDD>.db prior to rewriting.
          * Authoritative Data Preservation:
            - .unq (Unique & Synonym Dictionary): Identified as non-reconstructible authoritative master data; strictly exempted from derived index cleanup, preserved live, and snapshot-copied to <table_name>-v<ver>-<date>.unq.
            - .del (Soft-Deleted Records Archive): Detected via exist_table($table, 'del'), backed up, and all archived records migrated to ABR v5.
            - .aut (Audit Trail Log): Detected via exist_table($table, 'aut'), backed up, and user modification history migrated to ABR v5.
            - .cnt (Read Counter): Detected via exist_table($table, 'cnt'), backed up, and live counter state preserved.
          * Clean Re-indexing via insert_list: Cleans up all derived and legacy indexes (qw(inx src fld fac slg srt jinx jfld jsrc)) and rebuilds table data and indexes atomically through $adb->insert_list.

        - [COMMAND LINE UTILITY] Automated Migration Script (bin/update_tables.pl):
          * Added CLI migration script bin/update_tables.pl supporting --all, --table=<names>, --dbase=<dir>, --force, and --help options.
          * Provides granular per-table progress reporting and comprehensive post-migration summary (tables processed, upgraded, already up-to-date, record counts, and companion file backups).

        - [TRANSACTION SAFETY, SEARCH & CSV COMPATIBILITY]
          * Transact Journal Escaping: Escaped \n, \r, \x1e, and \\ in binary record payloads logged to .txn journals, preventing binary length bytes from splitting WAL records or corrupting field boundaries.
          * CSV Line Preservation: In tie2csv and vacuum, exported CSV records using _db_encode_legacy so exported files remain clean single-line human-readable text.
          * Unindexed Search Word Extraction: Fixed search_table in unindexed mode to decode record fields before passing them to get_words, preventing binary length bytes from corrupting search tokens.

        - [QUERY ENGINE & INDEX PERFORMANCE OPTIMIZATIONS] Adaptive Binary Intersect & Pure-Index Candidate Filtering:
          * Adaptive Binary Search Pruning (bin_crop): Introduced dynamic thresholding between XS unpack probing and 8-byte aligned O(log N) binary search (substr). For massive posting lists (e.g. 600K records), eliminates huge Perl scalar allocation...
          * Pure-Index Candidate Probing (search_table): Refactored multi-value/range filtering (e.g. 27-year date intervals) to evaluate candidate IDs directly against sorted 8-byte aligned index buffers via binary search, completely eliminating slo...
          * Batch Unindexed Fallback: Replaced iterative table_readid loops in unindexed search fallbacks with single-pass read_list batching, eliminating repetitive open/close file descriptor syscalls.

        - [TEST COVERAGE & PACKAGING]
          * Added comprehensive test suite t/amberdb_update_table.t covering multi-era format decoding, update_table migration, backup naming, .del/.aut/.cnt/.unq handling, and update_all batch discovery.
          * Updated MANIFEST to include bin/update_tables.pl and new test suites.
          * All 47 test files (440 assertions) passing with 100% success rate.

5.23.2  2026-09-03
        - [LOCALE ENGINE & MULTILINGUAL ARCHITECTURE] 10th Language - Global Base (gb) and Default Locale:
          * Introduced Global Base (gb) as the 10th supported language and new universal default/fallback locale (replacing en).
          * Implemented comprehensive multilingual Latin character preservation in alphabet_chars across European, Turkish, Nordic, French, German, Spanish, and Slavic-Latin alphabets.
          * Added cross-lingual, accent-tolerant search regex mapping (regex_map) matching accented and unaccented variations (e.g. cafe matches café, munchen matches münchen, seker matches şeker).
          * Implemented canonical accent folding in accent_map for high-recall inverted search indexing (.src).
          * Added lossless Unicode ligature conversion in ascii_map (ß->ss, æ->ae, œ->oe, ı->i, ø->o, ł->l, đ->d, ð->d, þ->th, ə->e) for clean URL slug and ASCII ID generation.

Changes  view on Meta::CPAN

          * Updated main AmberDB abstract to "High-performance embedded NoSQL database engine for Perl".

5.23.0  2026-09-02
        - [ARCHITECTURE & ID SIMPLIFICATION] Pure 64-bit Binary Engine & Deprecation of 8-Byte ASCII IDs:
          * Deprecated 8-byte ASCII IDs (a8) across binary index structures in favor of pure 64-bit Big-Endian unsigned integer packing (Q>*), guaranteeing O(1) binary slicing with fixed 8-byte record strides.
          * Simplified bin_encode() and bin_decode() by removing fragile \0 byte auto-detection and string unpacking heuristics.
          * Removed legacy id_type schema attribute across all modules and test suites; standard relational tables strictly enforce positive integer numeric IDs.
        - [STORAGE & PER-TABLE SIMPLE MODE] Hybrid Multi-Model Architecture with use_simple => 1:
          * Introduced per-table use_simple => 1 attribute allowing key-value tables with arbitrary string keys up to 255 bytes (UUIDs, session tokens, emails, slugs, etc.) alongside standard relational tables.
          * Preserved canonical physical table path ($dbase_dir/table/$table.db) for use_simple tables in standard database mode.
          * Selective schema sanitization: tables with use_simple => 1 strip indexing, columnar, and caching definitions (blocks, match_block, search_block, facet_block, sort_block, record_index, use_cache, cache_ttl) to achieve zero index/cache I/O ...
          * Enabled keep_deleted => 1 archiving (.del) on use_simple tables.
        - [RDBM & INTEGRITY] Foreign Key and Cross-Table Isolation:
          * Prohibited standard relational tables from binding foreign keys (RDBM) to use_simple tables in rdbm_target() and _resolve_field_value().
        - [TESTS & VERIFICATION] Dedicated Test Coverage:
          * Added comprehensive test suite t/amberdb_table_use_simple.t verifying arbitrary string keys, schema sanitization, zero index file creation, keep_deleted archiving, and RDBM isolation.
          * Cleaned up legacy id_type occurrences across all 41 test files and concurrency stress tests (xt/amberdb_concurrency_stress.t).
        - [DISTRIBUTION & METADATA] Decoupled Internal Module Versions and CPAN no_index:
          * Stripped redundant $VERSION definitions from 15 internal/engine-only modules (Base, Cache, Transact, Index, Index::Facet, Index::Junk, Locale::Currency, Locale::Lang::*) to streamline release maintenance.
          * Retained explicit $VERSION in public, standalone modules (AmberDB, AmberDB::Array, AmberDB::Date, AmberDB::Locale, AmberDB::Tools).
          * Added comprehensive META_MERGE no_index configuration in Makefile.PL covering internal directories, namespaces, and packages.

Changes  view on Meta::CPAN

5.22.0  2026-08-31
        - [REFACTOR] Codebase Cleanliness:
          * Removed redundant single-line helper is_rdbm_block() in favor of direct rdbm_target() calls across AmberDB::Index, AmberDB::Index::Junk, and AmberDB::Tools.
        - [RDBM & INTEGRITY] Hardened Foreign Table Auto-Registration in field_to_list():
          * Replaced decoupled table_autoid() and recs_put() sequence with atomic insert_id($target_table, 0, @record) to prevent ID race conditions.
          * Corrected foreign record column alignment so auto-registered values are accurately placed at block index $target_blk rather than hardcoded column 1.
          * Guaranteed that foreign table primary indexes (.inx) and secondary indexes (.unq, .src, .fac, .slg) are fully and consistently constructed upon auto-registration.
        - [SECURITY & VALIDATION] Universal ID Verification and Simple Mode Key Sanitization:
          * Eliminated opt-in config('id_check') requirement: every schema table now strictly enforces its schema id_type (positive integers for num, safe 8-byte chars for ascii).
          * Enforced scalar ID requirement: any reference (ARRAY ref, HASH ref, etc.) passed as a record ID is strictly rejected across all modes.
          * Introduced Safe Key Sanitization in Simple Mode: automatically trims leading/trailing whitespace, strictly rejects NUL bytes (\0) and control characters (\r, \n, \t, \x00-\x1F, \x7F) that corrupt Berkeley DB or CSV backups, and enforces a...
          * Integrated unconditional id_check across table_autoid(), read_id(), modify_id(), delete_id(), and exist_id().
        - [REFACTOR] Terminology and Codebase Refactoring: SEO -> Slug:
          * Completely replaced SEO terminology with Slug across the entire codebase, test suite, and documentation without legacy aliases:
            - Renamed methods: set_seourl() -> set_slug(), get_seourl() -> get_slug().
            - Renamed schema attributes: seo_block -> slug_block, seo_max_len -> slug_max_len.
          * Renamed test suite t/amberdb_seo_facet_bulk.t to t/amberdb_slug_facet_bulk.t.
        - [STORAGE] Standardized URL Slug Map File Extension: .rwt -> .slg:
          * Renamed binary slug map files from _0.rwt / _1.rwt to .slg (ID -> Slug) and .slg (Slug -> ID) across AmberDB, Index, Transact, and Tools.
          * Updated Tools->set_index() to reconstruct .slg files with zero data loss.
        - [DOCS] Caching Terminology Unification:

Changes  view on Meta::CPAN

        - Upgraded transaction engine specification to full ACID-Compliance with Strict Two-Phase Locking (Strict 2PL):
          * Enforced Lock-Before-Write and Lock-Before-Read ordering across insert_id, modify_id, and delete_id for true serializable isolation.
          * Introduced 'no_transact => 1' schema attribute and table_attr() support to exempt auxiliary tables from abort cascades while preserving LIFO rollback consistency.
        - Added comprehensive ACID architectural guarantees section to documentation (README.md, Turkish and English User Guides).
        - Clarified architectural distinction between high-throughput batch ETL imports and atomic business transactions.
        - Standardized file open error diagnostics and OS-level reporting ($!) across all core modules:
          * Added explicit OS error reporting ($!) to all open and tie failures in AmberDB, Base, Cache, Transact, and Tools.
          * Replaced silent schema open failure in AmberDB::Base::table_write with diagnostic cluck and graceful return.
          * Improved audit log error handling in AmberDB::auth_insert with cluck and record skipping.
        - Redesigned 2-Pillar Disaster Recovery and Native Backup Architecture:
          * Upgraded recs_back to continuous chronological time-series stream in 'backup/YYYY/YYYY-MM-DD.csv' eliminating folder clutter and ensuring zero-data-loss logging.
          * Added Tools->dump() for creating portable, compressed '.amberdb' archives packaging schemas (schema/*.table, schema/*.dbase), authoritative data files (tables/*.db, tables/*.del, tables/*.aut, tables/*.cnt, tables/*_*.str), and cryptograp...
          * Preserved native physical directory layout (schema/ and tables/) in .amberdb archives for 1-to-1 extraction and portability.
          * Added Tools->restore() with SHA-256 checksum verification, non-empty database safety checks, and automated deterministic binary index reconstruction via set_index.
          * Enhanced Tools->all_tables() with dual scalar/list context (grouped hashref vs. flat list) and automated 4-digit year directory discovery (e.g. 2024/, 2025/, 2026/).
          * Upgraded 'bin/convert_dbstore.pl' table discovery and added side-file reporting for .str string dictionaries alongside .del, .aut, and .cnt.
          * Introduced CLI utility 'bin/amberdb_backup.pl' for command-line database dump and disaster recovery operations.
          * Added comprehensive unit test suite t/amberdb_backup.t covering WAL streaming, .amberdb archiving, .dbase/.str preservation, and full database restore.
        - Updated POD documentation across AmberDB, AmberDB::Transact, and AmberDB::Tools modules.

5.02    2026-08-25
        - Initial public release prepared for CPAN and GitHub.
        - High-performance Berkeley DB (DB_File) flat-file database engine.
        - Packed 8-byte binary indexing pipeline for O(1) substr slicing.
        - High-speed index-assisted full-text search with phonetic and language normalization.
        - Tiered indexing (Active, Junk/Archived, and Hybrid AB/BA query modes).
        - Multi-dimensional Columnar Facet indexing (.fac) with high-efficiency bitsets.
        - Undo-journal transaction engine (transact_start, transact_end, transact_rollback) with automatic LIFO rollback.

MANIFEST  view on Meta::CPAN

t/amberdb-locale_03_num2text.t
t/amberdb-locale_04_sort.t
t/amberdb-locale_05_formatting.t
t/amberdb-locale_06_currency.t
t/amberdb-locale_07_search_regex.t
t/amberdb-locale_08_word_normalization.t
t/amberdb-locale_09_gb.t
t/amberdb.t
t/amberdb_array.t
t/amberdb_array_sort.t
t/amberdb_backup.t
t/amberdb_bin_crop.t
t/amberdb_bin_ops.t
t/amberdb_cache.t
t/amberdb_cli.t
t/amberdb_connect_lifecycle.t
t/amberdb_date.t
t/amberdb_dbase_inheritance.t
t/amberdb_ecommerce_facet.t
t/amberdb_encapsulation.t
t/amberdb_escape_encode.t

MANIFEST.SKIP  view on Meta::CPAN

# Version control
\B\.git\b
\B\.github\b
\B\.gitignore\b
\B\.gitattributes\b

# Temp, backup and editor files
~$
\.bak$
\.old$
\.tmp$
\.swp$
\#.*#$
\.\#.*
\b\.vscode\b
\b\.idea\b
\.code-workspace$
\blogs\b
\.log$
\bscratch\b
^\.backup\b
\b\.backup/
\b\.backup\b
\.rar$
\.zip$
\.7z$
^nul$
^NUL$
\bnul$
\bNUL$
[{}]
-scc_.*\.md$
^AGENTS\.md$

README.md  view on Meta::CPAN


- **Ultra High-Performance**: Leverages Berkeley DB (`DB_File`) hash storage with $O(1)$ binary slicing and configurable in-memory buffers.
- **JOIN-Free JSON-like Extensible Block Records**: Eliminates complex relational SQL `JOIN` overhead by storing hierarchical, extensible block records. Newly added blocks and attributes are automatically indexed on the fly for low-latency multi-dime...
- **Schema-Driven Dynamic Runtime Manipulation**: Table-specific schemas govern field validations, encodings, and index mappings. Schemas and values are fully mutable and can be modified dynamically at runtime without requiring table recreation or mi...
- **8-Byte Packed Binary Indexing**: Primary and secondary indexes use unified 8-byte packed binary buffers (`Q>*`), enabling $O(1)$ substring slicing, sub-millisecond pagination, and memory-efficient `keys_only` scalar pipelines (with `use_simple =>...
- **Intelligent & Locale-Aware Accent Search**: Advanced full-text search engine (`.src`) equipped with regional language and accent intelligence, phonetic devoicing (`b/d/g -> p/t/k`), circumflex/accent unfolding (`â/î/û -> a/i/u`), apostrophe su...
- **Columnar Facet Indexing (`.fac`)**: High-performance multi-dimensional facet filtering with index-level bitwise intersections and bidirectional string dictionaries (`.unq`) for e-commerce, catalogs, and large categorical datasets.
- **Multi-Tier Junk & Lifecycle Management**: Segregates active records from historical/archived data (`.db` master vs `.jnk` tier) with seamless single-pass hybrid queries (`jnktype => 'A' | 'B' | 'AB' | 'BA'`).
- **ACID-Compliant Undo-Journal Transactions**: Full ACID multi-table transactions with disk-backed undo journaling (`dbstore/journal/txn_*`), Strict Two-Phase Locking (Strict 2PL), automatic LIFO rollback upon failure or abnormal process exit, and o...
- **2-Pillar Disaster Recovery & Native `.amberdb` Archiving**: 
  - **Pillar 1 (Continuous Recovery Stream):** Automatic append-only audit stream in `backup/YYYY/YYYY-MM-DD.csv` capturing every `insert`, `modify`, and `delete`.
  - **Pillar 2 (Native Portable Archive):** Compressed, portable `.amberdb` archives containing schemas (`schema/*.table`, `schema/*.dbase`) and authoritative data files (`table/*.db`, `table/*.del`, `table/*.aut`, `table/*.cnt`, `table/*.unq`) with ...
- **Multi-Granularity Concurrency Control**: Non-blocking shared reads and exclusive writes at both table-level and individual record-level using OS-native `flock`.
- **ORM & Data Hydration (`inflate` / `deflate`)**: Native transformation between flat storage arrays and schema-mapped hash structures (`$adb->inflate` and `$adb->deflate`), including automatic RDBM foreign relationship resolution and repeating chil...
- **Granular Field Operations**: Direct field mutation without full record rewriting via `update_field`, positional child block insertion via `insert_field`, and safe targeted child deletion via `delete_field`.
- **Multilingual Locale Engine**: Out-of-the-box support for 10 languages (`gb` [default Global Base], `en`, `tr`, `de`, `fr`, `es`, `ja`, `ru`, `ar`, `az`) with language-specific case folding (e.g. Turkish `ı/I` and `i/İ`), cross-lingual accent fo...
- **High-Throughput 2-Phase Batch Operations**: High-performance batch ingestion pipeline (`insert_list`, `modify_list`, `delete_list`) opens master `.db` once for batch writing and executes single-pass index merging (`.inx`, `.src`, `.fld`, `.fac`, ...
- **Transparent Physical RAM-Disk Acceleration**: Integrated cross-platform orchestration (`tmpfs` Linux, `APFS` macOS, `ImDisk` Windows) across 4 operational tiers (0: Disk, 1: Hybrid Index-only, 2: Full RAM Mirror with dual-write, 3: Volatile pure ...

---

README.md  view on Meta::CPAN

├── table/                      ← Master Data & Derived Index Files
│   ├── catalog_product.db      ← Primary key-value data table (DB_File Hash)
│   ├── catalog_product.del     ← Soft-deleted records archive (keep_deleted)
│   ├── catalog_product.aut     ← User audit trail log (log_owner)
│   ├── catalog_product.cnt     ← View/hit counters (use_counter)
│   ├── catalog_product.unq     ← Bidirectional string-to-ID dictionary
│   ├── catalog_product.inx     ← Primary 8-byte packed ID and sort index
│   ├── catalog_product.fld     ← Inverted exact-match field index ("$blk:$val")
│   ├── catalog_product.src     ← Full-text keyword search index
│   └── catalog_product.fac     ← Columnar facet filter index
└── backup/                     ← Disaster Recovery & Archives
    └── 2026/
        ├── 2026-08-28.csv      ← Continuous time-series audit stream (Pillar 1)
        └── full_backup.amberdb ← Compressed native database archive (Pillar 2)
```

### File Extension Reference

AmberDB classifies physical file extensions into three operational tiers:

#### Authoritative Master Data (Source of Truth — Non-Reconstructible)
- **`.db`** — **Primary Document Table**: Berkeley DB master document storage (`DB_File` Hash).
- **`.del`** — **Soft-Deleted Archive**: Historical archive of soft-deleted records (`keep_deleted`).
- **`.aut`** — **User Audit Trail**: Chronological user action and modification audit log (`log_owner`).

README.md  view on Meta::CPAN


#### Derived Secondary Indexes (Reconstructible via `set_index`)
- **`.inx`** — **Primary & Sorted Record Index**: Binary array of active IDs, record count, highest ID, and pre-sorted order arrays (`sort_block`).
- **`.fld`** — **Inverted Match Index**: Block-level key-to-IDs inverted index (`match_block`).
- **`.src`** — **Full-Text Search Index**: Word-level token inverted index (`search_block`).
- **`.fac`** — **Facet Navigation Index**: Forward bitset index for faceted filter navigation (`facet_block`).
- **`.slg`** — **URL Slug Map**: Flat unified bidirectional slug map: `0:$rid` (ID→Slug) and `1:$slug` (Slug→ID).

#### Runtime & Backup Files
- **`.amberdb`** — **Native Database Archive**: Compressed tar archive with schemas, data files, and SHA-256 manifest.
- **`.csv`** — **Continuous WAL Stream**: Daily chronological append-only audit stream (`backup/YYYY/YYYY-MM-DD.csv`).
- **`.cnt`** — **View / Hit Counter**: High-throughput concurrent atomic counter store (`use_counter`).
- **`.tmp`** — **Disk Buffer File**: Disk staging buffer file under `dbstore/buffer/` (`buffer_write`).
- **`.lock`** — **Process Mutex Lock**: OS `flock` process synchronization lock file.

---

## Installation

### Standalone Windows Executable (`amberdb.exe`)

README.md  view on Meta::CPAN

}
```

### 7. Native Database Backup & Restore (`.amberdb`)

```perl
use AmberDB::Tools;

my $tools = AmberDB::Tools->new($adb);

# Create a compressed .amberdb backup archive
my $archive = $tools->dump();
# Archive created at: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb

# Restore archive with SHA-256 verification and automatic index rebuilding
my $result = $tools->restore(
    file    => "backup/2026/full_backup.amberdb",
    force   => 1, # Overwrite authorization for non-empty directories
    reindex => 1  # Automatically reconstruct .inx, .src, .fld, .fac, .slg
);
```

### 8. High-Throughput Batch Operations (Batch ETL & Ingestion)

When importing or updating hundreds or thousands of records from CSV, JSON, or external APIs, use the 2-phase batch methods (`insert_list`, `modify_list`, `delete_list`). These methods open the master `.db` file once and update all secondary indexes ...

```perl

README.md  view on Meta::CPAN

# Extract to C:\amberdb and add to User PATH (one-line PowerShell command):
[Environment]::SetEnvironmentVariable("Path", $env:Path + ";C:\amberdb", "User")
```

After restarting your terminal, use `amberdb` from any folder:
```cmd
amberdb setup
amberdb tables
amberdb read catalog_product 10
amberdb search catalog_product "laptop"
amberdb dump backup/catalog.amberdb
```

---

### 1. Unified Management CLI (`amberdb` / `amberdb.exe`)
The consolidated administrative CLI for infrastructure provisioning, database setup, RAM-disk orchestration, backups, and maintenance:
```bash
# Infrastructure setup (creates physical directories and baseline config)
amberdb setup
amberdb setup /path/to/dbstore

# RAM-disk management (Linux tmpfs, macOS APFS, Windows ImDisk)
amberdb setup ramdisk start 512M
amberdb setup ramdisk status
amberdb setup ramdisk stop

# Native backup (.amberdb dump and restore)
amberdb dump catalog file=backup/catalog.amberdb
amberdb restore file=backup/catalog.amberdb

# Storage layout & engine migrations
amberdb update storage --all
amberdb update version --check

# Re-index secondary binary indexes (.inx, .fld, .src, .slg)
amberdb reindex catalog
```

### 2. `bin/amberdb_daemon.pl` (Service Supervisor & Sync Daemon)

README.md  view on Meta::CPAN

  - [Komut Satırı Arayüzü (CLI) Kullanım Rehberi](wiki/TR-Guide-CLI.md)
  - [AmberDB Veritabanı Sistemi & Mimari Rehberi](docs/TR.AmberDB_Veritabani_Sistemi.md)
  - [AmberDB::Locale Kullanım Rehberi](docs/TR.AmberDB-Locale_Kullanim_Rehberi.md)
  - [AmberDB vs SQL Karşılaştırmalı Kullanım Rehberi](docs/TR.AmberDB-vs-SQL_Kullanim_Rehberi.md)
  - [AmberDB vs SQLite Kıyaslama Raporu (600K Film)](docs/TR.AmberDB-vs-SQLite_Benchmark.md)

---

## Running Tests

AmberDB includes an exhaustive test suite covering core operations, indexing, transactions, search, facets, locales, and backups, along with multi-process concurrency stress tests:

```bash
# Run standard unit & integration test suite (66 test files, 650+ assertions)
prove -l t/

# Run author & extended integration test suite (RAM-disk & multi-process stress)
prove -l xt/
# or directly:
perl -Ilib xt/amberdb_ramdisk.t
perl -Ilib xt/amberdb_concurrency_stress.t

bin/amberdb_cli.pl  view on Meta::CPAN

        }
        elsif ( $k =~ /^(?:force|f)$/ ) {
            $opt_force = $val ? 1 : 0;
        }
        elsif ( $k =~ /^(?:check|c)$/i ) {
            $method_args{check} = $val ? 1 : 0;
        }
        elsif ( $k =~ /^(?:all|a)$/i ) {
            $method_args{all} = $val ? 1 : 0;
        }
        elsif ( $k =~ /^(?:no-backup|no_backup)$/i ) {
            $method_args{no_backup} = $val ? 1 : 0;
        }
        elsif ( $k eq 'cpanm' ) {
            $method_args{cpanm} = $v;
        }
        elsif ( $k eq 'manifest' ) {
            $method_args{manifest} = $v;
        }
        elsif ( $k =~ /^(?:help|h|usage)$/i ) {
            $opt_help = 1;
            if ( defined $v && $v =~ /^(?:tr|en)$/i ) {

bin/amberdb_cli.pl  view on Meta::CPAN

        }
        elsif ( $arg =~ /^(?:force|f)$/i ) {
            $opt_force = 1;
        }
        elsif ( $raw =~ /^--?(?:check|c)$/i ) {
            $method_args{check} = 1;
        }
        elsif ( $raw =~ /^--?(?:all|a)$/i ) {
            $method_args{all} = 1;
        }
        elsif ( $raw =~ /^--?(?:no-backup|no_backup)$/i ) {
            $method_args{no_backup} = 1;
        }

        elsif ( $raw =~ /^--?time$/i ) {
            $opt_time = 1;
        }
        elsif ( !defined $opt_action ) {
            $opt_action = $arg;
        }
        else {
            push @pos_args, $arg;

bin/amberdb_cli.pl  view on Meta::CPAN

  amberdb setup [dir]                         # Initialize database skeleton in custom directory
  amberdb setup ramdisk [start|stop|status]   # RAM-disk drive management

Maintenance & Administrative Actions:
  amberdb update storage [--check] [--force]  # Directory & ABR v5 format migration
  amberdb update version [--check]            # MetaCPAN core version check / update
  amberdb reindex products                    # Rebuild all derived indexes
  amberdb check products                      # Physical file integrity check
  amberdb vacuum products                     # Compact and reclaim BDB storage space
  amberdb migrate products                    # Schema migration
  amberdb export products file=backup.csv     # Export table to CSV
  amberdb import products file=new.csv        # Import table from CSV
  amberdb dump products file=backup.tar.gz    # Create database backup archive
  amberdb restore file=backup.tar.gz          # Restore database from backup
  amberdb rename from=old to=new              # Rename table
  amberdb drop temp_tbl force=1               # Permanently drop table and indexes

General Options:
  --format=table|json|pretty|tsv|dumper       Output format (Default: table. Or trailing: json, tsv, dumper)
  --db=/path/to/dbstore                       Direct database directory without session
  --token=TOKEN                               Active session token
  --time, time, time=1                        Prints execution time at the bottom
  --dry-run                                   Simulates execution without writes
  --force                                     Required confirmation for destructive actions

bin/amberdb_cli.pl  view on Meta::CPAN

    else {
        # Central store under ~/.amberdb/<dbname>
        $target_data_dir = "$AMBERDB_HOME/$conn_db";
    }
    $target_data_dir = resolve_abs_path($target_data_dir);

    # Ensure target data directory and skeleton exist
    make_path(
        "$target_data_dir/table", "$target_data_dir/schema", "$target_data_dir/journal",
        "$target_data_dir/lock",  "$target_data_dir/session", "$target_data_dir/config",
        "$target_data_dir/backup", "$target_data_dir/ramdisk"
    );
    my $conf_file = "$target_data_dir/config/core.conf";
    if ( !-f $conf_file && open my $cfh, '>', $conf_file ) {
        print $cfh "site_id        $conn_db\nlanguage       tr\n";
        close $cfh;
    }

    $adb->set_datadir($target_data_dir);
    $adb->{_connect}->{database} = $conn_db;
    $adb->clear_cache('ramdisk');

bin/amberdb_cli.pl  view on Meta::CPAN

        print "     amberdb connect eticaretim\n";
        print "     amberdb 1245 tables\n\n";
        print "  3. Özel Klasör   : 'ad\@yol' formatı ile oturum açılır\n";
        print "     amberdb connect eticaretim\@./dbstore\n";
        print "=================================================================\n";
        exit 0;
    }

    # If custom directory passed (e.g. amberdb setup /var/data or amberdb setup my_dir)
    my $target_dir = resolve_abs_path($first_arg);
    my @subdirs = qw(table schema journal lock session config backup ramdisk);
    my @created;
    my @existing;
    make_path($target_dir) unless -d $target_dir;
    for my $sub (@subdirs) {
        my $dir_path = "$target_dir/$sub";
        if ( -d $dir_path ) {
            push @existing, $sub;
        }
        else {
            make_path($dir_path);

bin/amberdb_cli.pl  view on Meta::CPAN

    }
    $id //= 0;
    my $data  = delete $method_args{data};

    # Ensure database directory and skeleton exist on write
    my $db_dir = $adb->path('dbase_dir') || "dbstore";
    if ( !-d $db_dir || !-d "$db_dir/table" ) {
        make_path(
            "$db_dir/table", "$db_dir/schema", "$db_dir/journal",
            "$db_dir/lock",  "$db_dir/session", "$db_dir/config",
            "$db_dir/backup", "$db_dir/ramdisk"
        );
        my $conf_file = "$db_dir/config/core.conf";
        if ( !-f $conf_file && open my $cfh, '>', $conf_file ) {
            print $cfh "site_id        amberdb\nlanguage       tr\n";
            close $cfh;
        }
    }

    if ( !defined $data ) {
        $data = scalar keys %method_args ? \%method_args : [ map { parse_value($_) } @pos_args ];

bin/amberdb_cli.pl  view on Meta::CPAN

    output_result( { status => 'ok', action => 'insert_id', table => $table, id => $res }, $opt_format );
    exit 0;
}

# 9b. UPDATE_STORAGE
if ( $action eq 'update_storage' ) {
    my %opts;
    $opts{target_dir} = $adb->path('dbase_dir');
    $opts{force}      = $opt_force if $opt_force;
    $opts{check}      = $method_args{check} if exists $method_args{check};
    $opts{no_backup}  = $method_args{no_backup} if exists $method_args{no_backup};
    $opts{tables}     = $method_args{tables} // $method_args{table} // ( @pos_args ? join(',', @pos_args) : undef );
    $opts{manifest}   = $method_args{manifest} if exists $method_args{manifest};

    my $res = $tools->update_storage(%opts);
    output_result( $res, $opt_format ) if defined $opt_format;
    exit 0;
}

# 9c. UPDATE_VERSION
if ( $action eq 'update_version' ) {

bin/commands.md  view on Meta::CPAN

amberdb 1245 disconnect
```

### 4. Bakım ve Yönetim Eylemleri
```bash
amberdb reindex products
amberdb check products
amberdb vacuum products
amberdb export products file=products.csv
amberdb import products file=products.csv
amberdb dump products file=backup.tar.gz
amberdb restore file=backup.tar.gz force=1
amberdb rename from=old_table to=new_table
amberdb drop temp_table --force
```

docs/EN.About_AmberDB.md  view on Meta::CPAN

[Home](index.html) &nbsp;•&nbsp; [About](EN.About_AmberDB.html) &nbsp;•&nbsp; [Quick Start](index.html#quick-start) &nbsp;•&nbsp; [Tutorial](EN.AmberDB_User-Guide.html) &nbsp;•&nbsp; [Benchmark](EN.AmberDB-vs-SQLite_Benchmark.html) &nbsp;•&...

---

# About

**AmberDB** is an embedded NoSQL database engine for Perl that processes Array-based records supporting JSON-like nested and complex structures. Thanks to custom indexes generated through schema definitions, it performs block matching, querying, full...

It is built on the C code of Berkeley DB (`DB_File`) provided by the standard Perl package and does not use any other database engine. `DB_File` is Berkeley DB's low-level interface that provides key-value access via Perl. AmberDB constructs a data m...

---

## Why AmberDB?

AmberDB brings together diverse capabilities within a single Perl database engine:

* **No External Database Server:** Embedded directly into the application as an in-process object.
* **Array-Based Records:** Database records are Arrays (lists) supporting JSON-like nested structures.
* **Full CRUD Operations:** Fast, direct insert, read, update, and delete methods.

docs/EN.About_AmberDB.md  view on Meta::CPAN

$adb->modify_list("products", @records); # batch modify
$adb->delete_list("products", @ids);     # batch delete
```

For instance, when inserting a list of one thousand records using `insert_list`, AmberDB creates the records and their indexes in a single batch process. Similarly, a massive price list can be modified within a single batch operation. This makes batc...

---

## Portable Backups

Because AmberDB is file-based, it is fully portable. It also includes safer native database tools for backup and restoration.

A database can be exported into a portable `.amberdb` archive containing the authoritative database state and schema. Integrity is verified using SHA-256, and derived indexes are rebuilt during restoration. This makes it possible to migrate or archiv...

---

## Epilogue

AmberDB provides an experience close to **PostgreSQL + Elasticsearch + Redis**, achieving this at an extremely low footprint and cost. It requires no standalone server installation and embeds directly into your application. It consumes minimal system...

AmberDB is released under the **Artistic License 2.0**. Source code, documentation, and examples are available and actively maintained on CPAN and GitHub.

* **GitHub Issues:** [https://github.com/marufcetin/amberdb/issues](https://github.com/marufcetin/amberdb/issues)

docs/EN.AmberDB-vs-SQL_User-Guide.md  view on Meta::CPAN


Common tasks that require external libraries, database triggers, or external cache daemons in SQL stacks are natively built into AmberDB:

| Feature | Standard Solution in SQL | Built-in AmberDB Solution |
|---|---|---|
| **Automatic SEO URL Slugs** | Custom slugify code, database uniqueness check queries. | `slug_block => [1]` automatically creates collision-free slugs like `/product/sony-wh-1000xm5` upon insert/update (`get_slug`). |
| **User Audit Trail (Audit Log)** | Separate audit tables, database triggers, or ORM event hooks. | `log_owner => 1` logs every modification into `.aut`. Call `$adb->auth_view("table", $id)` for an instant HTML timeline. |
| **High-Speed In-Memory Caching & Ephemeral Data (RAM-Disk)** | Installing and managing external caching servers (`Redis` or `Memcached`) to bypass disk I/O bottlenecks, managing TCP network overhead and cache-sync logic. | Zero external servers or ...
| **Safe Soft-Delete** | Adding `is_deleted` column and remembering `WHERE is_deleted = 0` on every query. | `keep_deleted => 1` moves deleted records to a `.del` archive. Prevents data leaks with zero query filtering overhead. |

* **Reference:** [User Guide Section 12: URL Slugs](EN.AmberDB_User-Guide.html#12-automated-url-slug-management) · [Section 13: Shared RAM Cache](EN.AmberDB_User-Guide.html#13-unified-shared-ram-cache-db--inx-and-buffer) · [Section 17: Audit Trail]...

---

## 10. Quick Reference Cheat Sheet

A side-by-side mapping for common database operations:

| SQL Statement | AmberDB Method | AmberDB Usage Example |
|---|---|---|
| `INSERT INTO t VALUES (...)` | `insert_id` | `$id = $adb->insert_id("t", 0, @fields);` |

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

7. [Transaction Safety, ACID Guarantees, and Crash Recovery (Transactions)](#7-transaction-safety-acid-guarantees-and-crash-recovery-transactions)
8. [High-Throughput Batch Operations (Batch ETL & Ingestion)](#8-high-throughput-batch-operations-batch-etl--ingestion)
9. [Schema Configuration (.table & In-Memory)](#9-schema-configuration-table--in-memory)
10. [Database Group Structure (.dbase)](#10-database-group-structure-dbase)
11. [Smart Tiered (Hot / Cold Junk) Indexing](#11-smart-tiered-hot--cold-junk-indexing)
12. [Automated URL Slug Management](#12-automated-url-slug-management)
13. [Transparent Physical RAM-Disk Acceleration & In-Memory Storage](#13-transparent-physical-ram-disk-acceleration--in-memory-storage)
14. [Configuration and Deterministic Flag Management (`config`)](#14-configuration-and-deterministic-flag-management-config)
15. [Data Structures, Low-Level Table and Stream Operations](#15-data-structures-low-level-table-and-stream-operations)
16. [Faceted Search & Category Filters (Facet Engine)](#16-faceted-search--category-filters-facet-engine)
17. [User Audit Trail and Backup](#17-user-audit-trail-and-backup)
18. [Maintenance and Repair Tools (AmberDB::Tools)](#18-maintenance-and-repair-tools-amberdbtools)
19. [File Extensions Map](#19-file-extensions-map)
20. [Directory Structure](#20-directory-structure)
21. [Developer Best Practices and Recommendations](#21-developer-best-practices-and-recommendations)
22. [Full Working Example (Checkout & Stock Transaction Scenario)](#22-full-working-example-checkout--stock-transaction-scenario)
23. [Method Quick Reference Table](#23-method-quick-reference-table)
24. [Why Use AmberDB? (Comparison with SQL and SQLite)](#24-why-use-amberdb-comparison-with-sql-and-sqlite)
25. [Boundaries and Debated Topics (Physical Constraints vs. Conscious Architectural Choices)](#25-boundaries-and-debated-topics-physical-constraints-vs-conscious-architectural-choices)

---

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

---

### 5.2 Flexible & Arbitrary Record IDs (No 8-Byte Limit)

The standard mode **8-byte limit** and **strict ASCII/numeric format constraints** are relaxed in Simple Mode (`id_check` accepts arbitrary scalar keys and applies safe key sanitization):

- **Emails and Special Characters:** `user@example.com`, `api:v1:user:1005`
- **Long Tokens and UUIDs:** `sess_99999_abcdef_1234567890_extra_long_token` (up to 255 bytes)
- **Hyphenated Codes and Prefixes:** `TR-2026-08-31-INVOICE-001`
- **Unicode / Multilingual Keys:** `prod_özellik_kırmızı_xl`
- **Safe Key Sanitization:** Automatically trims leading/trailing whitespace (`trim_space`); strictly rejects NUL bytes (`\0`), control characters (`\r`, `\n`, `\t`), and references (ARRAY/HASH refs) to protect Berkeley DB C layers and CSV backup int...
- **Auto-ID Flexibility:** Custom IDs are not constrained to be strictly greater than `lastid`.

```perl
$adb->insert_id( 'sessions', 'user@example.com', 'Active', 'Chrome', time() );
my @sess = $adb->read_id( 'sessions', 'user@example.com' );
```

---

### 5.3 Data Operations (CRUD & Bulk)

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

if ($@) {
    $adb->transact_error( 'sessions', $@ );
}
my $txn = $adb->transact_end(); # Auto-rollbacks on error, commits if clean
```

---

### 5.6 Continuous Daily Backup Logs (`recs_back`)

Because text backup is schema-independent, **daily audit and continuous recovery streaming (`recs_back`)** is fully active in Simple Mode.

In accordance with Simple Mode's flat directory structure, no separate `backup/` or `YYYY/` subfolder is created. Every `insert_id` (`add`), `modify_id` (`edit`), and `delete_id` (`del`) operation is logged directly to **`$dbase_dir/YYYY-MM-DD.csv`**...

```text
2026-08-31 14:30:00    admin    add     sessions    sess_token_99999    Active\x1f192.168.1.50
2026-08-31 14:31:15    admin    edit    sessions    sess_token_99999    Closed\x1f192.168.1.50
2026-08-31 14:32:00    admin    del     sessions    sess_token_99999    
```

- To disable backup logging for volatile caches, configure `cfg => { no_backup => 1 }` or `$adb->config(no_backup => 1)`.
- Custom backup targets can be set via `path => { backup_dir => "/custom/backup/path" }`.

---

### 5.7 RAM-Disk Architecture & Caching in Simple Mode

In standard mode, AmberDB manages RAM-disk staging via schema `use_ramdisk => 2` rules.

**In Simple Mode, RAM-disk utilization is direct and flexible:**  
Since Simple Mode requires no schema files, creating a high-performance in-memory cache or session store simply involves binding a second AmberDB instance directly to the RAM-disk / tmpfs mount:

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

# 1. Persistent disk instance (For durable storage)
my $db_disk = AmberDB->new(
    path => { dbase_dir => "/var/data/app/dbstore/table" },
    cfg  => { use_simple => 1 },
);

# 2. RAM-Disk instance (Zero-latency in-memory cache/session store)
# (Linux: /dev/shm or tmpfs, Windows: ImDisk, macOS: APFS RAM-Disk /Volumes/AmberDB_RAM)
my $db_ramdisk = AmberDB->new(
    path => { dbase_dir => "/dev/shm/amberdb_cache" },
    cfg  => { use_simple => 1, no_backup => 1 }, # Disable backup for pure transient cache
);

# In-memory reads and writes at nanosecond speed:
$db_ramdisk->insert_id( "sessions", $session_token, $user_id, time() );
my @sess = $db_ramdisk->read_id( "sessions", $session_token );
```

Benefits of this dual-instance design:
- In-memory tables run without disk I/O bottlenecks.
- Persistent tables remain safely on durable physical storage.

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

| **Schema Files (`.table`, `.dbase`)** | Required & Enforced | None / Schemaless |
| **Arbitrary & Long Record IDs** | 8-Byte / Strict ASCII Limits | **Completely Unrestricted** |
| **Direct CRUD (`insert_id`, `read_id`)** | $O(1)$ | **$O(1)$ (Max Throughput)** |
| **Bulk Operations (`insert_list`, etc.)** | Supported | Supported |
| **Table Scan (`read_all`)** | Binary `.inx` or Direct | Direct Streaming Scan |
| **Pagination (`limit`) & `keys_only`** | Supported | Supported |
| **In-Memory Sorting (`sort => 2`)** | Supported | Supported |
| **Field Matching (`field_fetch`)** | Indexed `.fld` $O(1)$ | Sequential Streaming Scan |
| **Word Search (`search_table`)** | Inverted Index `.src` | Collation Streaming Scan |
| **ACID Transactions (`transact_*`)** | Supported (Index Undo) | **Supported (Raw Undo)** |
| **Continuous Daily Backup (`recs_back`)** | Supported (`backup/YYYY/`) | **Supported (Same Directory `YYYY-MM-DD.csv`)** |
| **Secondary Indexes (`.inx, .fld, .src, .slg, .fac`)** | Generated & Maintained | **Disabled (Zero Index Cost)** |
| **URL Slug Mapping (`.slg`)** | Auto Generated | Disabled |
| **Audit Logs (`.aut`) & Archive (`.del`)** | Schema-Driven | Disabled |
| **Directory Hierarchy** | `table/`, `schema/`, `backup/`, etc. | **Flat Single Directory (`$dbase_dir/<table_name>.db`)** |
| **Secondary Indexes (`.inx, .fld, .src, .slg, .fac`)** | Generated & Maintained | **Disabled (Zero Index Cost)** |
| **URL Slug Mapping (`.slg`)** | Auto Generated | Disabled |
| **Audit Logs (`.aut`) & Archive (`.del`)** | Schema-Driven | Disabled |
| **Directory Hierarchy** | `table/`, `schema/`, `backup/`, etc. | **Flat Single Directory (`$dbase_dir/<table_name>.db`)** |

---

## 6. Indexing and Search Engine

AmberDB maintains structured binary index files based on the schema configuration.

### 6.1 Index Types

| Extension | Index Type | Description |

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN


### 9.1 Database and Table Directory Layout

AmberDB stores tables, indexes, and schema definitions in dedicated physical directories under the configured `dbstore` root:

| Directory | Purpose |
|---|---|
| `dbstore/table/` | Base data (`.db`) and binary indexes (`.inx`, `.fld`, `.src`, `.fac`, `.slg`) |
| `dbstore/schema/` | Schema files (`.table`) and group configs (`.dbase`) |
| `dbstore/config/` | Plain-text `.conf` configuration files, `connect.pl` authentication profile |
| `dbstore/backup/` | Daily CSV audit backups (`dbgun/YYYYMMDD/`) |
| `dbstore/journal/`| Transaction undo WAL logs and Tier 4 write-behind queues (`txn_*`, `sync_ramdisk`) |
| `dbstore/lock/` | Process and table-level `flock` lock files when RAM-disk is unmounted (`*.lock`) |
| `dbstore/session/`| Process session token files when RAM-disk is unmounted (`*.sess`) |

> [!NOTE]
> **RAM-Disk Directory Structure:** When an OS RAM-disk is mounted (e.g. `R:/amberdb_$dbname` on Windows, `/dev/shm/amberdb_$dbname` on Linux, `/Volumes/amberdb_$dbname` on macOS), subdirectories (`table/`, `schema/`, `config/`, `lock/`, `session/`, ...

> [!IMPORTANT]
> **Directory Structure Compatibility Note:** The only manual action required when upgrading legacy projects is to rename your database directory's `dbstore/scheme/` folder to **`dbstore/schema/`**. All programmatic path resolutions and API calls are...

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

| `use_counter` | `0 / 1` | `0` | `usecnt` | Enables automated hit/view read counters in `.cnt` files. |
| `parent_table` | `string` | `""` | - | Parent table name for vertical partitioning (child table shares the same primary ID). |
| `force` | `0 / 1` | `0` | - | When `1`, `insert_id` overwrites existing records rather than failing (Replace mode). |
| `min_char` | `integer` | `2` | `minchar` | Minimum word length for full-text search indexing (1, 2, or 3). |
| `stop_word` | `string` | `""` | `nextkey` | Stop-words excluded from full-text search indexing (e.g., `"the and for with"`). |
| `repeat_ids` | `integer` | `undef` | - | Target block number where extracted child item IDs are consolidated. |
| `repeat_start` | `integer` | `undef` | - | Starting block index for dynamic repeating child rows (order items, cart lines). |
| `view_block` | `ARRAY` | `[]` | - | Priority block numbers displayed in UI / CMS listing views. |
| `use_menu` | `0 / 1` | `1` | - | Controls display of the table in admin panel navigation menus. |
| `no_transact` | `0 / 1` | `0` | - | Exempts table from transactional rollback error propagation. |
| `no_backup` | `0 / 1` | `0` | - | Disables daily CSV user audit logging for this table. |

---

### 9.7 Block (Field) Definitions, 8 Core Field Types, UI Inputs, and Validation Reference

Each block definition inside the `blocks` array supports the following attributes:

#### 9.7.1 Core Block Attributes

| Attribute | Type | Description | Example |

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

---

## 14. Configuration and Deterministic Flag Management (`config`)

Runtime behavior can be tuned and safely configured via the `$adb->config()` method:

```perl
# Bulk or single configuration assignment (Recommended)
$adb->config(
    no_write     => 1,            # Read-only maintenance mode: block all writes
    no_backup    => 1,            # Disable daily CSV audit logging for all tables
    use_simple   => 1,            # Direct unindexed mode: bypasses secondary index generation
    keys_only    => 1,            # read_all returns IDs only
    ramdisk_size => '1024M',      # RAM-Disk (tmpfs/APFS/ImDisk) size (Default: 512M)
);

# Single scalar getter:
my $no_write = $adb->config('no_write');

# Bulk getter (returns a safe shallow copy):
my $cfg = $adb->config();

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

```perl
# Retrieve user audit history as formatted HTML
my $history_html = $adb->auth_view("catalog_product", 5001);
print $history_html;
# Output:
#     add     2026-08-14 10:15    admin_user
#     edit    2026-08-14 11:30    editor_user
```

### 17.2 Continuous Recovery Stream (`YYYY-MM-DD.csv`)
AmberDB automatically appends every `insert`, `modify`, and `delete` operation into a clean, chronological time-series stream in `backup/YYYY/YYYY-MM-DD.csv`.

Each entry is tab-separated (`\t`) using the standard format:
`[Timestamp] \t [User] \t [Action] \t [Table] \t [Record ID] \t [Packed Values]`

To disable this backup stream:
* **In Table Schema (Per-Table):** Add `no_backup => 1` in the table schema to disable logging for that specific table only.
* **Globally via Config (All Tables):** Set `$adb->config(no_backup => 1);` to disable logging across all tables.

### 17.3 Native Database Archive (`.amberdb` Dump & Restore)
AmberDB packages all schemas (`schema/*.table`, `schema/*.dbase`) and authoritative data files (`tables/*.db`, `tables/*.del`, `tables/*.aut`, `tables/*.cnt`) alongside cryptographically verified SHA-256 checksums in a single compressed, portable **`...

Derived index files (`.inx`, `.src`, `.fld`, `.fac`) are intentionally excluded to keep archives compact and ensure future-proof portability; `restore` deterministically rebuilds all indexes via `set_index`.

```perl
use AmberDB;
use AmberDB::Tools;

my $adb   = AmberDB->new(path => { dbase_dir => "./dbstore" });
my $tools = AmberDB::Tools->new($adb);

# 1. Create full database backup archive (.amberdb)
my $archive = $tools->dump();
# Output: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb

# 2. Export specific tables as a focused snapshot archive
$tools->dump(
    file   => "backup/2026/catalog_backup.amberdb",
    tables => ["catalog_product", "catalog_category"]
);

# 3. Restore database archive and automatically rebuild all indexes
$tools->restore(
    file    => "backup/2026/catalog_backup.amberdb",
    force   => 1, # Overwrite confirmation for non-empty target directories
    reindex => 1  # Automatically reconstruct binary indexes from source data
);
```

#### CLI Command-Line Utility (`bin/amberdb_backup.pl`)
```bash
# Dump entire database to default archive
perl bin/amberdb_backup.pl --dump --file backup/2026/full_backup.amberdb

# Dump specific tables only
perl bin/amberdb_backup.pl --dump --tables products,orders

# Restore database archive with integrity checks and automatic reindexing
perl bin/amberdb_backup.pl --restore --file backup/2026/full_backup.amberdb --force
```

---

## 18. Maintenance and Repair Tools (AmberDB::Tools)

`AmberDB::Tools` provides utilities for reindexing, table vacuuming, and data migration:

```perl
use AmberDB;

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN

│   ├── catalog_product.src      ← Full-text search index
│   ├── catalog_product.fac      ← Facet filtering index
│   ├── catalog_product.unq      ← String dictionary
│   ├── catalog_product.slg      ← Bidirectional URL slug map ("0:$id", "1:$slug")
│   ├── catalog_product.aut      ← Audit trail
│   └── catalog_product.del      ← Soft-deleted records
├── ramdisk/                     ← RAM-Disk Mount & Storage (Linux tmpfs, macOS APFS, Windows ImDisk)
├── buffer/                      ← Transient Disk Buffer / Staging Files
├── journal/                     ← Active Transaction Journals (txn_*)
├── pids/                        ← Lock Files
└── backup/                      ← Daily CSV Backups
```

---

## 21. Developer Best Practices and Recommendations

1. **Use `insert_list` for Bulk Ingestion:** When adding hundreds of records, use `insert_list` instead of looping over `insert_id`. Batch mode writes all records in a single file session and rebuilds indexes in one pass.
2. **Wrap Multi-Step Writes in `transact_start`:** Always wrap inventory deductions, checkout sequences, or multi-table balance updates inside transactions.
3. **Index Only Required Fields:** Only assign fields to `match_block` or `search_block` if they are actively queried to minimize disk write overhead.
4. **Always Handle Pagination Return Signatures Correctly:** When passing `$limit > 0` to `read_all`, `field_fetch`, or `search_table`, remember that the first returned value is `$total_count` integer. Never unpack into a single array (`my @records =...

docs/EN.AmberDB_User-Guide.md  view on Meta::CPAN


### 24.4 Direct Inverted Key Lookups (Zero Query Planner Overhead)
In SQL, running `SELECT id FROM orders WHERE customer_id = 'A'` requires parsing, query plan evaluation, cost optimization, and virtual machine execution.

In AmberDB, `field_fetch` is a direct hash key lookup on Berkeley DB returning packed binary buffers. Query planning overhead is zero.

### 24.5 Built-in Lifecycle and Domain Features
- **Automatic URL Slug Management:** When titles or categories change, clean slugs like `/products/laptop-pro-m3` and conflict resolution suffixes are generated automatically.
- **Audit Trails (.aut):** User identity, action type (`add`, `edit`, `del`), and timestamps are recorded without extra tables.
- **Safe Soft Deletion (.del):** Deleted records are archived safely and can be inspected or restored.
- **Zero Configuration & Portability:** Copying the database directory creates a complete, standalone backup that can run on any Perl-enabled system.

---

## 25. Boundaries and Debated Topics (Physical Constraints vs. Conscious Architectural Choices)

In database design, every architectural decision serves a specific optimization goal. Certain characteristics that developers coming from traditional SQL environments might initially perceive as "constraints" or "omissions" are, in fact, **deliberate...

### 25.1 Physical and Environmental Boundaries (Out-of-Scope Scenarios)

The following scenarios lie outside the intended operational scope of an embedded, file-based database engine like AmberDB:

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

}
my $txn = $adb->transact_end(); # Hata varsa otomatik rollback, yoksa commit
```

---

### 5.6 Günlük Sürekli Yedekleme Günlükleri (Daily Backup Logs - `recs_back`)

Basit mod şemadan bağımsız olduğundan, **metin tabanlı sürekli denetim ve kurtarma akışı (`recs_back`)** basit modda da varsayılan olarak devrededir.

Basit modun düz dizin yapısı gereği ayrı bir `backup/` veya `YYYY/` alt klasörü oluşturulmaz; yapılan her `insert_id` (`add`), `update_id` (`edit`) ve `delete_id` (`del`) işlemi doğrudan tablolarla aynı dizinde bulunan **`$dbase_dir/YYYY-...

```text
2026-08-31 14:30:00    admin    add     sessions    sess_token_99999    Aktif\x1f192.168.1.50
2026-08-31 14:31:15    admin    edit    sessions    sess_token_99999    Kapali\x1f192.168.1.50
2026-08-31 14:32:00    admin    del     sessions    sess_token_99999    
```

- İstenirse `cfg => { no_backup => 1 }` veya `$adb->config(no_backup => 1)` ile yedekleme günlükleri devre dışı bırakılabilir.
- Özel bir yedek dizini tanımlanmak istendiğinde `path => { backup_dir => "/harici/yedek/yolu" }` verilebilir.

---

### 5.7 Basit Modda RAM-Disk Mimarisi ve Önbellekleme

AmberDB standart modda RAM-disk hızlandırmasını şemadaki `use_ramdisk => 2` kuralı ile yönetir.

**Basit modda ise RAM-disk kullanımı çok daha doğrudan ve esnektir:**  
Basit mod şemaya ihtiyaç duymadığından, yüksek performanslı bir bellek önbelleği / geçici oturum deposu oluşturmak için ikinci bir AmberDB basit nesnesi doğrudan RAM-disk yoluna bağlanır:

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

# 1. Kalıcı disk nesnesi (Kalıcı veriler için)
my $db_kalici = AmberDB->new(
    path => { dbase_dir => "/var/data/eticaret/dbstore/table" },
    cfg  => { use_simple => 1 },
);

# 2. RAM-Disk nesnesi (Sıfır gecikmeli hızlı oturum/önbellek tabloları için)
# (Linux: /dev/shm veya tmpfs, Windows: ImDisk, macOS: APFS RAM-Disk /Volumes/AmberDB_RAM)
my $db_ramdisk = AmberDB->new(
    path => { dbase_dir => "/dev/shm/amberdb_cache" },
    cfg  => { use_simple => 1, no_backup => 1 }, # Önbellek için yedekleme kapatılabilir
);

# RAM üzerinde nanosaniye hızında oturum okuma/yazma:
$db_ramdisk->insert_id( "oturumlar", $session_token, $user_id, time() );
my @oturum = $db_ramdisk->read_id( "oturumlar", $session_token );
```

Bu çift nesneli mimari sayesinde:
- RAM-disk üzerindeki tablolar disk I/O darboğazına takılmadan bellek hızında çalışır.
- Kalıcı tablolar ana depolama alanında güvenle tutulmaya devam eder.

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

| **Şema Dosyaları (`.table`, `.dbase`)** | Zorunlu / Kullanılır | Yok / Şemasız |
| **Özel ve Uzun ID'ler (UUID, E-posta, vb.)** | 8 Bayt / ASCII Kısıtlı | **Tamamen Serbest** |
| **Tekil CRUD (`insert_id`, `read_id`, vb.)** | $O(1)$ | **$O(1)$ (Maksimum Hız)** |
| **Toplu İşlemler (`insert_list`, vb.)** | Desteklenir | Desteklenir |
| **Tüm Tablo Okuma (`read_all`)** | `.inx` veya doğrudan | Doğrudan Dosya Taraması |
| **Sayfalama (`start`/`limit`) ve `keys_only`** | Desteklenir | Desteklenir |
| **Bellek İçi Sıralama (`sort => 2`)** | Desteklenir | Desteklenir |
| **Alana Göre Filtre (`field_fetch`)** | `.fld` İndeksli $O(1)$ | Sıralı Dosya Taraması |
| **Tam Metin Arama (`search_table`)** | `.src` Ters İndeksli | Sıralı Dosya Taraması (Türkçe Normalizasyonlu) |
| **ACID İşlemler (`transact_*`)** | Desteklenir (İndeks Geri Alma Dahil) | **Desteklenir (Ham Veri Geri Alma)** |
| **Sürekli Yedekleme Akışı (`recs_back`)** | Desteklenir (`backup/YYYY/`) | **Desteklenir (Aynı Dizinde `YYYY-MM-DD.csv`)** |
| **İkincil İndeksler (`.inx, .fld, .src, .fac`)** | Oluşturulur ve Güncellenir | **Oluşturulmaz (Sıfır İndeks Maliyeti)** |
| **URL Slug Rewrite (`.slg`)** | Otomatik Üretilir | Devre Dışı |
| **Denetim İzi (`.aut`) ve Arşiv (`.del`)** | Şema Kuralına Göre Tutulur | Devre Dışı |
| **Dizin Yapısı** | `tables/`, `schema/`, `backup/` vb. | **Düz Kök Dizin (`$dbase_dir/<tablo>.db`)** |
| **İkincil İndeksler (`.inx, .fld, .src, .fac`)** | Oluşturulur ve Güncellenir | **Oluşturulmaz (Sıfır İndeks Maliyeti)** |
| **URL Slug Rewrite (`.slg`)** | Otomatik Üretilir | Devre Dışı |
| **Denetim İzi (`.aut`) ve Arşiv (`.del`)** | Şema Kuralına Göre Tutulur | Devre Dışı |
| **Dizin Yapısı** | `tables/`, `schema/`, `backup/` vb. | **Düz Kök Dizin (`$dbase_dir/<tablo>.db`)** |

---

## 6. İndeksleme ve Arama Mekanizması

AmberDB, tablolara hızlı erişim sağlamak için veriyi şemada tanımlanan kurallara göre ikili (binary) indeks dosyalarına yazar.

### 6.1 İndeks Türleri

| Dosya Uzantısı | İndeks Türü | Açıklama |

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN


### 9.1 Veritabanı ve Tablo Dizin Yapısı

AmberDB tabloları, şemaları ve geçici/kalıcı dosyaları, belirlenen `dbstore` ana veri dizini altında fiziksel klasörlere ayrılarak saklanır:

| Dizin | Görevi |
|---|---|
| `dbstore/table/` | Kalıcı `.db` ana veri, `.inx` kayıt ve sıralama indeksi, `.fld` eşleştirme, `.src` arama, `.fac` facet, `.slg` slug dosyaları |
| `dbstore/schema/` | Kalıcı `.table` tablo şemaları ve `.dbase` grup yapılandırma dosyaları |
| `dbstore/config/` | Kalıcı `.conf` düz metin ayar ve konfigürasyon dosyaları, `connect.pl` profil dosyası |
| `dbstore/backup/` | Günlük CSV denetim yedekleri (`dbgun/YYYYMMDD/`) |
| `dbstore/journal/`| İşlem geri alma (WAL) ve Tier 4 gecikmeli yazma günlükleri (`txn_*`, `sync_ramdisk`) |
| `dbstore/lock/` | RAM-disk bağlı değilken süreç ve tablo seviyesi kilit dosyaları (`*.lock`) |
| `dbstore/session/`| RAM-disk bağlı değilken süreç oturum dosyaları (`*.sess`) |

> [!NOTE]
> **RAM-Disk Dizin Yapısı:** İşletim sisteminde RAM-disk bağlandığında (Windows'ta `R:/amberdb_$dbname`, Linux'ta `/dev/shm/amberdb_$dbname`, macOS'ta `/Volumes/amberdb_$dbname`), RAM-disk kök dizini altında dinamik olarak `table/`, `schema...

> [!IMPORTANT]
> **Dizin Yapısı Uyumluluk Notu:** Eski projelerden yükseltme yaparken yapmanız gereken tek fiziksel işlem; veritabanı dizininizdeki `dbstore/scheme/` klasörünün adını **`dbstore/schema/`** olarak yeniden adlandırmaktır. Kod ve API taraf...

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

---

## 14. Yapılandırma ve Deterministik Bayrak Yönetimi (`config`)

AmberDB'nin çalışma modunu değiştirmek ve yapılandırma bayraklarını güvenli bir şekilde yönetmek için `$adb->config()` metodu kullanılır:

```perl
# Toplu veya tekli yapılandırma ataması (Önerilen)
$adb->config(
    no_write     => 1,            # Bakım modu: Tüm yazma işlemlerini engelle
    no_backup    => 1,            # Tüm tablolar için günlük CSV denetim yedeklerini kapat
    use_simple   => 1,            # İndekssiz doğrudan yazım modu (İkincil indeksler devre dışı bırakılır)
    keys_only    => 1,            # read_all çağrılarında sadece ID'leri döndür
    ramdisk_size => '1024M',      # RAM-Disk (tmpfs/APFS/ImDisk) boyutu (Varsayılan: 512M)
);

# Tekil okuma:
my $no_write = $adb->config('no_write');

# Toplu okuma (Güvenli kopya döner):
my $cfg = $adb->config();

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

```perl
# Bir kaydın kimler tarafından ne zaman değiştirildiğini HTML olarak alma
my $gecmis_html = $adb->auth_view("catalog_product", 5001);
print $gecmis_html;
# Çıktı:
#     add     2026-08-14 10:15    admin_maruf
#     edit    2026-08-14 11:30    editor_ali
```

### 17.2 Sürekli Değişiklik Akışı (Continuous Recovery Stream - `YYYY-MM-DD.csv`)
AmberDB yapılan her `insert`, `modify` ve `delete` işlemini kronolojik zaman-serisi olarak `backup/YYYY/YYYY-MM-DD.csv` dosyasına otomatik olarak ekler (append-only).

Her satır tab ayrılmış (`\t`) olarak şu sütun yapısında yazılır:
`[Zaman Damgası] \t [Kullanıcı] \t [İşlem] \t [Tablo] \t [Kayıt ID] \t [Paketlenmiş Değerler]`

Bu akışı devre dışı bırakmak için:
* **Tablo Şemasında (Tablo Bazlı):** Şema dosyasına `no_backup => 1` eklenirse sadece o tablo için yedekleme kapatılır.
* **Genel Düzeyde (Tüm Tablolar):** `$adb->config(no_backup => 1);` tanımlanırsa tüm tablolar için yedekleme kapatılır.

### 17.3 Native Veritabanı Arşivi (`.amberdb` Dump & Restore)
AmberDB, tüm şemaları (`schema/*.table`, `schema/*.dbase`) ve otoriter veri dosyalarını (`tables/*.db`, `tables/*.del`, `tables/*.aut`, `tables/*.cnt`) SHA-256 doğrulama özetleriyle birlikte fiziksel dizin yapısıyla birebir örtüşen sıkı...

Türetilmiş indeks dosyaları (`.inx`, `.src`, `.fld`, `.fac`, `.slg`) boyuttan tasarruf etmek için arşiv içine konmaz; `restore` esnasında şema kurallarına göre `set_index` ile deterministik olarak sıfırdan üretilir.

```perl
use AmberDB;
use AmberDB::Tools;

my $adb   = AmberDB->new(path => { dbase_dir => "./dbstore" });
my $tools = AmberDB::Tools->new($adb);

# 1. Tüm veritabanının tam yedeğini alma (.amberdb)
my $arsiv = $tools->dump();
# Çıktı: dbstore/backup/2026/amberdb_2026-08-28_180000.amberdb

# 2. Belirli tabloların snapshot yedeğini alma
$tools->dump(
    file   => "backup/2026/katalog_yedek.amberdb",
    tables => ["catalog_product", "catalog_category"]
);

# 3. Yedeği geri yükleme ve tüm indeksleri otomatik inşa etme
$tools->restore(
    file    => "backup/2026/katalog_yedek.amberdb",
    force   => 1, # Var olan tabloların üzerine yazma izni
    reindex => 1  # İndeksleri sıfırdan üret
);
```

#### CLI Komut Satırı Aracı (`bin/amberdb_backup.pl`)
```bash
# Veritabanını yedekleme
perl bin/amberdb_backup.pl --dump --file backup/2026/tam_yedek.amberdb

# Belirli tabloları yedekleme
perl bin/amberdb_backup.pl --dump --tables products,orders

# Yedeği güvenli şekilde geri yükleme
perl bin/amberdb_backup.pl --restore --file backup/2026/tam_yedek.amberdb --force
```

---

## 18. Bakım ve Onarım Araçları (AmberDB::Tools)

Veritabanı indekslerini sıfırdan yeniden oluşturmak, veri doğrulaması yapmak veya tabloları optimize etmek için `AmberDB::Tools` kullanılır:

```perl
use AmberDB;

docs/TR.AmberDB_Veritabani_Sistemi.md  view on Meta::CPAN

│   ├── catalog_product.src      ← Tam metin arama indeksi
│   ├── catalog_product.fac      ← Facet filtreleme indeksi
│   ├── catalog_product.unq      ← Alan metin sözlüğü
│   ├── catalog_product.slg      ← Çift yönlü URL slug haritası ("0:$id", "1:$slug")
│   ├── catalog_product.aut      ← Denetim logu
│   └── catalog_product.del      ← Silinen kayıtlar
├── ramdisk/                     ← RAM-Disk Paylaşımlı Bellek ve Hızlı Depolama (Linux tmpfs, macOS APFS, Windows ImDisk)
├── buffer/                      ← Geçici Disk Buffer / Staging Dosyaları
├── journal/                     ← Aktif Transaction Günlükleri (txn_*)
├── pids/                        ← Dosya ve Kayıt Kilitleri
└── backup/                      ← Günlük CSV Yedekleri
```

---

## 21. Geliştirici Tavsiyeleri ve En İyi Pratikler

1. **Toplu Veri Girişinde `insert_list` Kullanın:** Yüzlerce kaydı tek tek döngüde `insert_id` ile eklemek yerine tek seferde `insert_list` ile ekleyin; disk I/O ve indeksleme süresi 10 kat hızlanacaktır.
2. **Kritik İş Mantıklarında `transact_start` Kullanın:** Stok düşme, bakiye güncelleme ve sipariş onaylama gibi adımları mutlaka transaction bloğu içine alın.
3. **Şemalarda Gereksiz Blokları İndekslemeyin:** Yalnızca filtrelenecek alanları `match_block`, aranacak alanları `search_block` olarak tanımlayın.
4. **Sayfalama Dönen Değer İmzasını Doğru Karşılayın:** `read_all`, `field_fetch` ve `search_table` metotlarında `$limit > 0` verildiğinde dönen listenin ilk elemanının `$toplam` tamsayısı olduğunu unutmayın. Asla `my @kayitlar = $a...

lib/AmberDB.pm  view on Meta::CPAN

    if ($is_async_write) {
        $self->ramdisk_mark_dirty( $file_path, $rid, 1 );
    }
    elsif ( $use_ramdisk == 4 && $is_txn ) {
        $self->ramdisk_unmark_dirty( $file_path, $rid );
    }

    $self->table_close($target_file);
    unless ($is_txn) { $self->flock_close( $tableid, $rid ); }

    # for index actions and backup
    @record = ( $rid, @record );

    # Invalidate cached table keys and count in memory
    $self->set_cache( $tableid, 'keys', undef );
    $self->set_cache( $tableid, 'count', undef );

    unless ( $is_async_write || $use_ramdisk == 3 ) {
        $self->recs_back( "add", $tableid, \@record );
    }

lib/AmberDB.pm  view on Meta::CPAN

    $self->slug_add( $idx_path, $table_info, $tableid, \@batch );
    $self->suggest_add( $idx_path, $table_info, $tableid, \@batch );

    unless ($is_async_write) {
        if ( $table_info->{log_owner} ) {
            foreach my $rec (@batch) {
                $self->auth_write( $tableid, $table_path, "add", $rec->[0] );
            }
        }

        unless ( $self->config('no_backup') || $table_info->{no_backup} ) {
            $self->recs_back( "add", $tableid, @batch );
        }
    }

    return \%statu;
}

# Replace the DB record with new data.
# ------------------------------------------------
sub update_id {

lib/AmberDB.pm  view on Meta::CPAN


    $self->table_close($file_path) unless ( $is_async_write || $use_ramdisk == 3 );
    $self->table_close($ram_file) if $ram_file;
    unless ($is_txn) { $self->flock_close( $tableid, $rid ); }

    # Cache invalidate
    $self->clear_cache($tableid, $rid);

    my @new_rec = ( $rid, @record );

    # text backup record.
    unless ( $is_async_write || $use_ramdisk == 3 ) {
        $self->recs_back( "edit", $tableid, \@new_rec )
          or cluck "[DB_TIE] Backup error (edit). $tableid\n";
    }

    if ( $self->config('use_simple') || ( $table_info && $table_info->{use_simple} ) ) {
        $self->_invalidate_child_ramdisk_tables($tableid);
        return $rid;
    }

lib/AmberDB.pm  view on Meta::CPAN

    $self->unique_modify( $idx_path, $table_info, \@pairs );

    $self->slug_modify( $idx_path, $table_info, $tableid, \@pairs );

    if ( $table_info->{log_owner} && !$is_async_write ) {
        foreach my $pair (@pairs) {
            $self->auth_write( $tableid, $table_path, "edit", $pair->[0] );
        }
    }

    unless ( $is_async_write || $self->config('no_backup') || $table_info->{no_backup} ) {
        $self->recs_back( "edit", $tableid, map { $_->[2] } @pairs );
    }

    return \%statu;
}

# Standard SQL/CRUD aliases
sub modify_id   { shift->update_id(@_) }
sub modify_list { shift->update_list(@_) }

lib/AmberDB.pm  view on Meta::CPAN

        $self->ramdisk_unmark_dirty( $file_path, $rid );
    }

    $self->table_close($file_path) unless ( $is_async_write || $use_ramdisk == 3 );
    $self->table_close($ram_file) if $ram_file;
    unless ($is_txn) { $self->flock_close( $tableid, $rid ); }

    # Cache invalidate
    $self->clear_cache($tableid, $rid);

    # Text backup record
    unless ( $is_async_write || $use_ramdisk == 3 ) {
        $self->recs_back( "del", $tableid, [ $rid, "" ] )
          or cluck "[DB_TIE] Backup error (del). $tableid\n";
    }

    # Move to archive if keep_deleted enabled
    if ( $table_info->{keep_deleted} && !$is_async_write ) {
        (         $self->table_write($del_path)
              and $self->recs_put( [ $del_path, $tableid ], [ $rid, $record ] )
              and $self->table_close($del_path) )

lib/AmberDB.pm  view on Meta::CPAN

    }
    elsif ( $table_info->{use_facet} ) {
        $self->facet_del( $idx_path, $table_info, \@batch );
    }

    $self->unique_del( $idx_path, $table_info, \@batch );

    $self->slug_del( $idx_path, $table_info, $tableid, \@batch );

    unless ( $is_async_write || $use_ramdisk == 3 ) {
        # Operations: auth, backup
        if ( $table_info->{log_owner} ) {
            foreach my $rec (@batch) {
                $self->auth_write( $tableid, $table_path, "del", $rec->[0] );
            }
        }

        unless ( $self->config('no_backup') || $table_info->{no_backup} ) {
            $self->recs_back( "del", $tableid, map { [ $_->[0], "" ] } @batch );
        }
    }

    $self->_invalidate_child_ramdisk_tables($tableid);
    return \%statu;
}

# my $ok = $adb->insert_links($tableid, [rid1, lnk1], [rid2, lnk2]);
# Writes alias link bindings into .lnk file.

lib/AmberDB.pm  view on Meta::CPAN

                if ( $mirror_path && $mirror_path ne $table_path ) {
                    $self->index_del( [ $mirror_path, $tableid, 1 ], \@keys_to_del );
                }
            }
        }
    }

    return $del_count;
}

# Writes add|edit|del operation to daily CSV backup audit stream (backup/YYYY/YYYY-MM-DD.csv).
# Exits silently if no_backup is set (globally or in table schema).
# my $ok = $adb->recs_back("add|edit|del", $tableid, @records);
# ------------------------------------------------
sub recs_back {

    my ( $self, $action, $tableid, @records ) = @_;

    ( $action and $tableid and scalar @records ) or return;

    # Global config check: disables backup for all tables
    return 1 if $self->config('no_backup');

    # Table schema check: no_backup => 1 in table schema or volatile Tier 3 RAM-disk
    my $table_info = $self->table_info($tableid);
    return 1 if $table_info->{no_backup} || ( $table_info->{use_ramdisk} // 0 ) == 3;

    my $user = $self->config('user') || 'system';

    $tableid =~ s/[:\/\\]/--/g;

    my $backup_base = $self->path('backup_dir')
      || ( $self->path('dbase_dir') ? $self->path('dbase_dir') . "/backup" : "backup" );
    my $year     = $self->year;
    my $month    = $self->month;
    my $day      = $self->day;
    my $date_iso = "$year-$month-$day";
    my $time_str = $self->date_str;

    my $backup_file;
    if ( $self->config('use_simple') ) {
        $backup_file = "$backup_base/$date_iso.csv";
    }
    else {
        my $year_dir = "$backup_base/$year";
        $self->make_path($year_dir);
        $backup_file = "$year_dir/$date_iso.csv";
    }

    open my $YAZ, ">>:encoding(UTF-8)", $backup_file
      or do {
        cluck "[DB_BACKUP] Cannot open backup file $backup_file: $!\n";
        return;
      };

    foreach my $record (@records) {
        ref($record) eq "ARRAY" or $record = [$record];
        my $rid     = $record->[0];
        my $bac_val = $self->db_encode( @{$record}[ 1 .. $#$record ] );
        print $YAZ "$time_str\t$user\t$action\t$tableid\t$rid\t$bac_val\n";
    }
    close $YAZ;

lib/AmberDB.pm  view on Meta::CPAN

=head1 SUBMODULE ARCHITECTURE & INHERITANCE

C<AmberDB> is built as a unified coordinator that incorporates all functionality from specialized submodules via inheritance (C<use parent>). When you instantiate an C<AmberDB> object (C<$adb>), all methods from the following submodules are directly ...

=over 4

=item * B<L<AmberDB::Date>> - Compact chronological ID getters (C<day_id>, C<second_id>, C<month_id>), date string parsing (C<str2dateid>, C<dateid2str>), range generation (C<day_range>), ISO week numbers (C<dateid2week>), and relative offset calcula...

=item * B<L<AmberDB::Locale>> - Multilingual text processing, locale-aware casing (C<uc>, C<lc>, C<ucfirst>), Unicode Collation (UCA) sorting (C<sort>), ASCII transliteration (C<to_ascii>), number-to-words / cheque conversion (C<num2text>), number fo...

=item * B<L<AmberDB::Tools>> - Enterprise database utilities: backup (C<.amberdb> tar archive with SHA-256 integrity), atomic restore, CSV migration (C<tie2csv>, C<csv2tie>), vacuum compaction, and reindexing.

=back

All submodules (except C<AmberDB::Tools> which takes an C<$adb> handle) can also be instantiated and used independently in standalone scripts.

=head1 TABLE NAMING CONVENTIONS

AmberDB enforces a strict, deterministic lowercase snake_case table naming convention:

=over 4

lib/AmberDB.pm  view on Meta::CPAN

=head2 Key Characteristics of Simple Mode

=over 4

=item * B<Arbitrary & Flexible Keys:> The 8-byte ASCII limit and numeric constraints are bypassed. Keys can be emails (C<user@example.com>), UUIDs, long tokens, or Unicode/multilingual strings.

=item * B<Flat Directory Structure:> All tables reside directly under C<dbase_dir> (e.g. C<$dbase_dir/table.db>). No C<table/> or C<schema/> subfolders are required.

=item * B<Rich Nested Structures:> Records can store nested array and hash references (ARRAY/HASH) directly.

=item * B<Continuous Daily Backup Logs:> Text-based continuous daily WAL/CSV logs (C<recs_back>) automatically record all C<add>, C<edit>, and C<del> operations directly into C<$dbase_dir/YYYY-MM-DD.csv> alongside database tables (can be silenced wit...

=item * B<ACID Transactions:> Full multi-table transaction support with atomic rollback (restoring raw records in the C<.db> file).

=item * B<Streaming Queries & Sorting:> Methods like C<read_all>, C<field_fetch>, and C<search_table> operate via direct sequential streaming scans with full support for pagination (C<offset>/C<limit>, with legacy C<start>), C<keys_only>, and in-memo...

=item * B<Zero-Latency RAM-Disk Caching:> Simple mode instances can be initialized directly on RAM-disk mount points (Linux C</dev/shm>, macOS C</Volumes/AmberDB_RAM>, or Windows ImDisk C<R:>) to provide nanosecond-speed transient session and cache s...

=back

=head2 Simple Mode Example

lib/AmberDB.pm  view on Meta::CPAN


  # 3. Read record (O(1))
  my @sess = $adb->read_id('sessions', 'user@example.com');

  # 4. Search and filter without indexes
  my ($count, @active) = $adb->field_fetch('sessions', 1, 'Active', 0, 10);

  # 5. Volatile in-memory simple store
  my $ram_db = AmberDB->new(
      path => { dbase_dir => "/dev/shm/amberdb_tokens" },
      cfg  => { use_simple    => 1, no_backup => 1 },
  );
  $ram_db->insert_id('tokens', $token_id, $user_id, time());

=head1 RAM-DISK ACCELERATION

AmberDB features a built-in, transparent physical RAM-disk acceleration engine (Linux C<tmpfs>, macOS C<APFS RAM-Disk> via C<hdiutil>, or Windows C<ImDisk>). It enables high-traffic tables and hot indexes to operate at pure volatile memory speeds whi...

=head2 What is it?

RAM-disk acceleration routes file I/O for database tables directly to an operating system in-memory RAM-disk filesystem (Windows ImDisk C<R:/amberdb_$dbname>, Linux tmpfs C</dev/shm/amberdb_$dbname>, or macOS APFS C</Volumes/amberdb_$dbname>). Unlike...

lib/AmberDB.pm  view on Meta::CPAN

    my $count = $adb->field_count("catalog_product", 2, 45);
    # 1452

When C<$value> is omitted, counts all values indexed for that block:

    my $all_counts = $adb->field_count("catalog_product", 2);
    # { 45 => 1452, 68 => 21, 712 => 85, ... }

=head2 search_table($table_id, $query, [\%options])

It performs searches matching query terms using the full-text C<.src> index (or a sorted table scan backup method if unindexed). C<search_table> uses the C<AmberDB::Locale> module. It features advanced language normalization according to the selected...

B<IMPORTANT (Return Signature Convention):>
When C<$limit> is passed and C<E<gt> 0> (paginated), C<search_table> returns C<($total_count, @records)> where the first scalar is the total matching count integer. When C<$limit> is omitted or C<0> (unpaginated), it returns C<@records> directly. Unp...

    # 1. Unpaginated (returns array of record arrayrefs directly)
    my @records        = $adb->search_table("catalog_product", "wireless headphones");
    my @sorted_records = $adb->search_table("catalog_product", "headphones", { sort => -5 });

    # 2. Paginated (first element is total matching count integer)
    my ($total_count, @search) = $adb->search_table("catalog_product", "headphones", { offset => 0, limit => 20 });

lib/AmberDB/Base.pm  view on Meta::CPAN


# $self->set_datadir("/path/to/dbase")
# ------------------------------------------------
sub set_datadir {
    my ( $self, $dbase_dir ) = @_;

    $dbase_dir or return;

    # declarations (persistent storage only; ramdisk directories are dynamically provisioned on mount)
    my @dirs = qw(
      dbase_dir table_dir schema_dir backup_dir
      buffer_dir journal_dir lock_dir session_dir config_dir
    );
    foreach my $dir (@dirs) {
        $self->{_path}->{$dir} //= "";
    }

    $self->{_path}->{dbase_dir} = $dbase_dir;

    # db_ext tanımlı ve "db" değil ise simple moduna al
    my $db_ext = $self->config('db_ext');
    if ( length($db_ext) && $db_ext ne "db" ) {
        $self->config( use_simple => 1 );
    }

    # do not proceed if simple mode
    if ( $self->config('use_simple') ) {
        $self->{_path}->{backup_dir}  = $dbase_dir;
        $self->{_path}->{buffer_dir}  = $dbase_dir;
        $self->{_path}->{journal_dir} = $dbase_dir;
        $self->{_path}->{schema_dir}  = $dbase_dir;
        $self->{_path}->{table_dir}   = $dbase_dir;
        $self->{_path}->{lock_dir}    = "$dbase_dir/lock";
        $self->{_path}->{session_dir} = "$dbase_dir/session";
        $self->{_path}->{config_dir}  = "$dbase_dir/config";
        return 1;
    }

    $self->{_path}->{journal_dir} = "$dbase_dir/journal";
    $self->{_path}->{backup_dir}  = "$dbase_dir/backup";
    $self->{_path}->{buffer_dir}  = "$dbase_dir/buffer";
    $self->{_path}->{schema_dir}  = "$dbase_dir/schema";
    $self->{_path}->{table_dir}   = "$dbase_dir/table";
    $self->{_path}->{lock_dir}    = "$dbase_dir/lock";
    $self->{_path}->{session_dir} = "$dbase_dir/session";
    $self->{_path}->{config_dir}  = "$dbase_dir/config";

    # Auto-load connect.pl if present in datadir
    my $conn_file = "$dbase_dir/config/connect.pl";
    if ( -f $conn_file ) {

lib/AmberDB/Base.pm  view on Meta::CPAN

            $self->{_connect}->{database} = $cfg->{database};
        }
    }

    unless ( $self->config('test') || $self->config('no_mkdir') || $^C ) {
        if ( defined $dbase_dir && $dbase_dir ne "." && $dbase_dir ne "" ) {
            for my $dir (
                $self->{_path}->{dbase_dir},
                $self->{_path}->{table_dir},
                $self->{_path}->{schema_dir},
                $self->{_path}->{backup_dir},
                $self->{_path}->{buffer_dir},
                $self->{_path}->{journal_dir},
                $self->{_path}->{lock_dir},
                $self->{_path}->{session_dir},
            ) {
                if ( defined $dir && length($dir) && !$self->dir_exist($dir) ) {
                    $self->make_path($dir);
                }
            }
        }

lib/AmberDB/Base.pm  view on Meta::CPAN

            if ( $key eq 'user' ) {
                $self->{_connect}->{username} = $val;
            }
        }
    }

    return $self;
}

# my $dbase_dir = $adb->path("dbase_dir");         # getter
# $adb->path({ backup_dir => "$tmpdir/backup" });  # setter
# $adb->path( custom_path => "/opt/data" );        # setter
# ------------------------------------------------
sub path {
    my ( $self, @args ) = @_;

    # 1. No arguments: return shallow copy of all path mappings
    if ( !@args ) {
        return { %{ $self->{_path} || {} } };
    }

lib/AmberDB/Base/Schema.pm  view on Meta::CPAN


our $VERSION = '5.26.6';
my $CREATED  = '2026-09-06';

# Whitelist of inheritable group-level configuration keys from .dbase to member tables
our %INHERITABLE_DBASE_KEYS = map { $_ => 1 } qw(
    use_ramdisk   ramdisk_ttl   use_cache   cache_ttl
    keep_deleted  log_owner     use_counter
    use_section   section       use_year    year
    use_language  lang
    no_transact   no_backup     table_dir   use_junk
    use_simple
);

# =====================================================================
# SCHEMA FIELD NORMALIZATION, VALIDATION & TYPE CONVERSION
# =====================================================================

# Normalizes and validates a single field value according to schema block definition.
# Usage:
#   my $clean_val = $adb->enc_field($blk_def, $val);

lib/AmberDB/Base/Schema.pm  view on Meta::CPAN

    if ( ( $schema->{use_ramdisk} // 0 ) == 3 ) {
        delete @{$schema}{
            qw(
              blocks match_block search_block view_block facet_block filter_block
              suggest_block suggest_join
              slug_block sort_block sort_fields use_facet facet_rules use_junk junk_rules
              record_index repeat_start repeat_ids field_rules keep_deleted
            )
        };
        $schema->{use_simple}  = 1;
        $schema->{no_backup}   = 1;
        $schema->{no_transact} = 1;
        $schema->{ramdisk_ttl} = 300 unless defined $schema->{ramdisk_ttl};
    }

    # ------------------------------------------------------------------------
    # Pipeline Step 2: Simple Mode Stripping
    # In simple key-value mode, selectively strip columnar, indexing, relational,
    # and caching definitions to enforce lightweight operation.
    # ------------------------------------------------------------------------
    if ( $schema->{use_simple} && ( $schema->{use_ramdisk} // 0 ) != 3 ) {

lib/AmberDB/Base/Schema.pm  view on Meta::CPAN

    $table_path =~ s/\//-/g;

    my $table_str = "";

    # Scalar keys
    my @scalar_keys = qw(
      name record_index keep_deleted log_owner parent_table
      use_menu use_simple force use_ramdisk ramdisk_ttl use_cache use_alias
      use_counter use_facet stop_word min_char
      use_junk cache_ttl repeat_ids repeat_start
      no_transact no_backup table_dir
    );
    foreach my $key (@scalar_keys) {
        next unless exists $tbl->{$key};
        next unless defined $tbl->{$key} && $tbl->{$key} ne "";
        ( my $val = $tbl->{$key} ) =~ s/"/\\"/g;
        $table_str .= "\t$key => \"$val\",\n";
    }

    # Array keys
    my @array_keys = qw(

lib/AmberDB/Tools.pm  view on Meta::CPAN

AmberDB::Tools - Database maintenance, CLI reindexing, migrations, and bulk conversion toolset

=head1 SYNOPSIS

  use AmberDB;
  use AmberDB::Tools;

  my $adb   = AmberDB->new(path => { dbase_dir => "/path/to/dbstore" });
  my $tools = AmberDB::Tools->new($adb);

  # 1. Create portable .amberdb database backup archive
  my $archive = $tools->dump();

  # 2. Restore database archive with integrity validation and reindexing
  my $result  = $tools->restore(file => "backup.amberdb", force => 1);

  # 3. Rebuild all indexes for a single table (.inx, .src, .fld, .fac)
  $tools->set_index("catalog_product");

  # 4. Rebuild only specific index components
  $tools->set_search("catalog_product");  # Rebuild full-text search index
  $tools->set_fields("catalog_product");  # Rebuild field exact match index
  $tools->set_filters("catalog_product"); # Rebuild facet forward filter index
  $tools->set_sort("catalog_product");    # Rebuild binary pre-sorted sequences in .inx

lib/AmberDB/Tools.pm  view on Meta::CPAN

=head1 DESCRIPTION

C<AmberDB::Tools> provides maintenance, native disaster recovery archiving (C<dump>/C<restore>), batch utility functions for rebuilding indexes, populating full-text search inverted files, compiling forward facet filter dictionaries, generating binar...

Functionality is organized into focused sub-modules:

=over 4

=item * B<AmberDB::Tools::Index>: Index generation routines (C<set_index>, C<set_readall>, C<set_search>, C<set_fields>, C<set_filters>, C<set_rwlnkall>, C<set_sort>, C<index_alltables>, C<check_readall>, C<check_search>).

=item * B<AmberDB::Tools::Maintain>: Maintenance, export/import, and backup routines (C<dump>, C<restore>, C<vacuum>, C<del_table>, C<tie2csv>, C<csv2tie>, C<dir_tables>, C<all_tables>).

=item * B<AmberDB::Tools::Update>: Table format conversions, storage upgrades, and engine updating (C<update_table>, C<update_all>, C<replace_tablename>, C<replace_blockdata>, C<db_simple>, C<convert_tables>, C<update_storage>, C<update_version>).

=back

All operations can also be invoked directly from the terminal via C<amberdb> CLI (e.g. C<amberdb update storage>, C<amberdb update version>, C<amberdb reindex products>).

=head1 CONSTRUCTOR

=head2 new($adb, [%options])

lib/AmberDB/Tools.pm  view on Meta::CPAN

=head1 METHODS

=head2 dump([%options])

Creates a compressed, portable C<.amberdb> archive file (gzipped tar archive) containing table and database schemas (C<schema/*.table>, C<schema/*.dbase>), native database data files (C<table/*.db>, C<table/*.del>, C<table/*.aut>, C<table/*.cnt>), an...

Options:

=over 4

=item * C<file>: Custom output file path (defaults to C<backup/YYYY/amberdb_YYYY-MM-DD_time.amberdb>).

=item * C<tables>: Array reference of table IDs to include (defaults to all tables in database).

=item * C<table>: Single table ID to export as a focused snapshot.

=back

  my $archive = $tools->dump();
  my $archive = $tools->dump(tables => ["catalog_product", "orders_cart"]);

lib/AmberDB/Tools.pm  view on Meta::CPAN

=item * C<file>: Path to C<.amberdb> archive file (required).

=item * C<force>: Boolean (default 0). Must be set to 1 to overwrite existing tables in a non-empty database directory.

=item * C<reindex>: Boolean (default 1). Automatically executes C<set_index> for all restored tables.

=item * C<tables>: Array reference of specific table IDs to extract from the archive.

=back

  my $res = $tools->restore(file => "backup.amberdb", force => 1);

=head2 set_index($table_id, [@records])

Rebuilds all secondary and primary indexes for C<$table_id> based on its schema definition:

=over 4

=item * Primary key index (C<.inx>) via C<set_readall>

=item * Full-text search inverted indexes (C<.src>) via C<set_search>

lib/AmberDB/Tools.pm  view on Meta::CPAN

  $tools->set_sort("catalog_product");

=head2 convert_tables()

Scans the entire database directory, identifies all physical tables, and sequentially runs C<set_index> to rebuild and migrate packed binary indexes across the entire system. Returns a status hash reference.

  my $status = $tools->convert_tables();

=head2 update_table($table_id, [%options])

Scans an entire table record-by-record, detects legacy formats, creates a timestamped backup, and rewrites the table in native ABR v5 format while rebuilding indexes.

  my $res = $tools->update_table("catalog_product", force => 1);

=head2 update_storage([%options])

Migrates storage directories (C<scheme/> to C<schema/>, C<tables/> to C<table/>), converts legacy TSV tables to native ABR v5 format, rebuilds all indexes, and stamps C<config/storage_version.json>.

Options: C<check>, C<force>, C<no_backup>, C<tables>, C<target_dir>.

  $tools->update_storage(force => 1);

=head2 update_version([%options])

Queries the MetaCPAN API for newer releases of AmberDB, and optionally invokes C<cpanm> to upgrade the engine.

Options: C<check>, C<cpanm>.

  $tools->update_version(check => 1);

lib/AmberDB/Tools/Maintain.pm  view on Meta::CPAN

    my $adb = $self->{_adb} or return;

    $tableid or return;

    # file paths
    my $table_path = $adb->table_path($tableid);
    my $file_path  = "${table_path}.$adb->{db_ext}";
    my $csv_path   = "$table_path.csv";
    return unless -e $csv_path;

    # backup with timestamp if exists
    if ( -e "${table_path}.$adb->{db_ext}" ) {
        my $sec_id = $adb->second_id;
        rename( "${table_path}.$adb->{db_ext}",
            "${table_path}-$sec_id.$adb->{db_ext}" );
        unlink("${table_path}.$adb->{db_ext}");
    }

    # perform conversion operations
    my $i = 1;
    my @records;

lib/AmberDB/Tools/Maintain.pm  view on Meta::CPAN

    if ( $adb->config('use_simple') ) {
        my $ext = $adb->{db_ext} || "db";
        my @files = $adb->dir_files( $dbase_dir, "*.$ext", full_path => 0 );
        @all_tables = map { /^([a-z0-9_]+)\.\Q$ext\E$/i ? $1 : () } @files;
    }
    # 2. Standard Structured Mode: Multi-directory scan (table/ and year directories) for .db files
    else {
        my $tbl_dir = File::Spec->catdir( $dbase_dir, 'table' );
        push @all_tables, $adb->dir_files( $tbl_dir, "*.db", full_path => 0 ) if -d $tbl_dir;

        my %seen_dirs = ( "table" => 1, "schema" => 1, "backup" => 1, "lock" => 1, "ramdisk" => 1, "journal" => 1, "session" => 1, "config" => 1 );
        if ($year_dir) {
            my $yd_path = File::Spec->catdir( $dbase_dir, $year_dir );
            push @all_tables, $adb->dir_files( $yd_path, "*.db", full_path => 0 );
            $seen_dirs{$year_dir} = 1;
        }

        # Auto-discover any 4-digit year directories under $dbase_dir (e.g. 2024, 2025, 2026)
        if ( -d $dbase_dir ) {
            my @year_dirs = $adb->dir_files( $dbase_dir, qr/^\d{4}$/, full_path => 0, files_only => 0 );
            foreach my $yd (@year_dirs) {

lib/AmberDB/Tools/Maintain.pm  view on Meta::CPAN

    # 2. Flush and close open table handles for read consistency
    $adb->close_all();

    # 3. Determine output file path
    my $year     = $adb->year;
    my $month    = $adb->month;
    my $day      = $adb->day;
    my $date_iso = "$year-$month-$day";
    my $time_id  = $adb->second_id;

    my $backup_base = $adb->path('backup_dir')
      || ( $adb->path('dbase_dir') ? $adb->path('dbase_dir') . "/backup" : "backup" );
    my $year_dir = "$backup_base/$year";
    $adb->make_path($year_dir);

    my $outfile = $opts{file} || "$year_dir/amberdb_${date_iso}_${time_id}.amberdb";

    # Ensure parent directory for $outfile exists
    if ( my ($outdir) = $outfile =~ m{^(.*)[/\\]} ) {
        $adb->make_path($outdir);
    }

    my $tar = Archive::Tar->new();

lib/AmberDB/Tools/Maintain.pm  view on Meta::CPAN

        cluck "[DB_BACKUP] Failed to write archive $outfile: " . $tar->error() . "\n";
        return;
    }

    $self->{say} .= "Archive successfully written to $outfile (" . ( -s $outfile ) . " bytes)\n";
    return wantarray ? ( $outfile, $manifest ) : $outfile;
}

# Restores a .amberdb archive into target database, validates checksums,
# writes data files via native DB_File, and deterministically rebuilds all indexes.
# my $res = $tools->restore( file => 'backup.amberdb', [force => 1], [reindex => 1] );
# ---------------------------------------------------------------------
sub restore {
    my ( $self, @args ) = @_;
    my $adb = $self->{_adb} or return;

    my %opts;
    if ( @args == 1 && ref( $args[0] ) eq 'HASH' ) {
        %opts = %{ $args[0] };
    }
    elsif ( @args % 2 == 1 && !ref( $args[0] ) ) {

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

        print $out $buf;
    }
    close $in;
    close $out;
    return 1;
}

# ---------------------------------------------------------------------
# update_table($tableid, %options)
# Scans an entire table record-by-record, detects any legacy formats (2003-2026),
# creates a timestamped backup with detected dominant version, and rewrites the table
# directly in ABR v1 format preserving original IDs.
# Rebuilds indexes using ($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index($tableid).
# Preserves companion data files ('del', 'aut').
# ---------------------------------------------------------------------
sub update_table {
    my ( $self, $tableid, %opts ) = @_;
    my $adb = $self->{_adb} or return;
    $tableid or return;

    my $table_path = $adb->table_path($tableid);

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

    # Format modification timestamp: YYYY-MMDD
    my @mtime_parts = localtime( ( stat($file_path) )[9] || time() );
    my $mtime_str   = sprintf( "%04d-%02d%02d", $mtime_parts[5] + 1900, $mtime_parts[4] + 1, $mtime_parts[3] );

    # 1. Once mevcut tablodaki kayitlari tara
    my %format_counts;
    my $total                 = 0;
    my $already_current = 0;
    my $legacy_count    = 0;
    my @decoded_records;
    my $recovered_from_backup = 0;

    $adb->recs_scan( $file_path, sub {
        my ( $k, $v ) = @_;
        $total++;
        my $fmt = $adb->detect_tsv_format($v);
        $format_counts{$fmt}++;

        my @fields;
        if ( $fmt eq 'v5' ) {
            @fields = $adb->db_decode($v);

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

        else {
            @fields = $adb->tsv_decode( $v, $k );
            $legacy_count++;
        }
        push @decoded_records, [ $k, @fields ];
        } );
    $adb->table_close($file_path);

    # Eger aktif dosya bos ise (ornegin onceki basarisiz bir calismada yedeklenip bos kalmissa):
    # Dizin icindeki yedek dosyasini bulup kayitlari oradan kurtar
    my $backup_file;
    if ( $total == 0 ) {
        my ( $pdir ) = $file_path =~ m{^(.*)[/\\]};
        $pdir //= ".";
        my @cand_backups;
        if ( opendir( my $dh, $pdir ) ) {
            while ( my $f = readdir($dh) ) {
                next if $f eq '.' || $f eq '..';
                if ( $f =~ /^\Q$tableid\E-.*?\.\Q$ext\E$/i ) {
                    push @cand_backups, "$pdir/$f";
                }
            }
            closedir($dh);
        }
        @cand_backups = sort { ( -s $b ) <=> ( -s $a ) } @cand_backups;
        for my $cb (@cand_backups) {
            next unless -e $cb && -s $cb;
            my @b_records;
            my %b_counts;
            my $b_legacy = 0;
            $adb->recs_scan( $cb, sub {
            my ( $k, $v ) = @_;
                my $fmt = $adb->detect_tsv_format($v);
                $b_counts{$fmt}++;
                my @fields = ( $fmt eq 'v5' ) ? $adb->db_decode($v) : $adb->tsv_decode( $v, $k );
                $b_legacy++ if $fmt ne 'v5';
                push @b_records, [ $k, @fields ];
        } );
            $adb->table_close($cb);
            if (@b_records) {
                @decoded_records       = @b_records;
                %format_counts         = %b_counts;
                $total                 = scalar @b_records;
                $legacy_count          = $b_legacy;
                $already_current       = $total - $b_legacy;
                $recovered_from_backup = 1;
                $backup_file           = $cb;
                last;
            }
        }
    }

    # 2. Eslikci veri dosyalarini ('del', 'aut') tara ve formatlarini tespit et
    my %companion_info;
    my $companion_legacy_total = 0;
    for my $cext (qw(del aut)) {
        next unless $adb->exist_table( $tableid, $cext );

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

        }
        if ( $table_info->{match_block} && !-e "$table_path.fld" ) {
            $missing_indexes = 1;
        }
        if ( $table_info->{use_facet} && !-e "$table_path.fac" ) {
            $missing_indexes = 1;
        }
    }

    # Eger ana tablo ve tum eslikci dosyalar zaten guncelse, eksik indeks yoksa ve force istenmemisse
    if ( $legacy_count == 0 && $companion_legacy_total == 0 && !$opts{force} && !$recovered_from_backup && !$missing_indexes ) {
        $self->{say} .= "Table '$tableid' is already up to date in ABR v1 format ($already_current records).\n";
        my $res = {
            status          => 'already_current',
            table           => $tableid,
            total           => $total,
            updated         => 0,
            already_current => $already_current,
        };
        for my $cext (qw(del aut)) {
            if ( my $ci = $companion_info{$cext} ) {

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

    my $dom_ver = 'v4';
    my $max_c   = -1;
    for my $v (qw(v4 v3 v2 v1)) {
        if ( ( $format_counts{$v} // 0 ) > $max_c ) {
            $max_c   = $format_counts{$v} // 0;
            $dom_ver = $v;
        }
    }

    # 4. Ana tablo donusumu (gerekliyse veya force ise)
    if ( $legacy_count > 0 || $recovered_from_backup || $opts{force} ) {
        # Eger yedekten kurtarilmadiysa, aktif dosyayi nihai yedek adiyla yedekle
        if ( !$recovered_from_backup ) {
    my $backup_base = "$table_path-$dom_ver-$mtime_str";
            $backup_file = "$backup_base.$ext";
    my $counter     = 1;
    while ( -e $backup_file ) {
        $backup_file = "$backup_base-$counter.$ext";
        $counter++;
    }
    rename( $file_path, $backup_file ) or do {
        cluck "[DB_TOOL] Could not backup $file_path to $backup_file: $!\n";
        return { status => 'error', table => $tableid, error => "Backup failed: $!" };
    };
        }

        # Yeni dosyayi sifir temiz yazma modunda ac
        $adb->table_close($file_path);
        $adb->clear_cache($tableid);
        my $new_db = $adb->table_write($file_path);
        unless ($new_db) {
            return { status => 'error', table => $tableid, error => "Could not open $file_path for writing" };

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

        $adb->table_close($file_path);

        # Eski turetilen indeks dosyalarini temizle
        for my $iext (qw(inx src fld fac slg ajw ajn)) {
            my $idx_f = "$table_path.$iext";
            unlink $idx_f if -e $idx_f;
        }
    $adb->clear_cache($tableid);
    }
    else {
        $backup_file //= "None (already ABR v1)";
        $self->{say} .= "  - Main table is already in ABR v1 format ($already_current records).\n";
    }

    # 5. Eslikci veri dosyalarini ('del', 'aut') donustur (indeks insasindan once calisir)
    my %companion_stats;
    for my $cext (qw(del aut)) {
        my $ci = $companion_info{$cext};
        next unless $ci;
        my $cfile = $ci->{file};

        if ( $ci->{legacy} > 0 || $opts{force} ) {
            if ( $ci->{total} > 0 ) {
                $self->{say} .= "  - Migrating companion file ($cext: $ci->{total} records)...\n";
        my $c_backup_base = "$table_path-$dom_ver-$mtime_str.$cext";
        my $c_backup = $c_backup_base;
        my $cnt = 1;
        while ( -e $c_backup ) {
            $c_backup = "$table_path-$dom_ver-$mtime_str-$cnt.$cext";
            $cnt++;
        }
        rename( $cfile, $c_backup ) or next;

        my $new_cdb = $adb->table_write($cfile);
        if ($new_cdb) {
                    for my $crec ( @{ $ci->{decoded} } ) {
                my ( $ck, @cfields ) = @$crec;
                my $cv_new = $adb->db_encode(@cfields);
                $new_cdb->put( $adb->utf_encode("$ck"), $adb->utf_encode("$cv_new") );
            }
            $adb->table_close($cfile);
                    $companion_stats{"${cext}_migrated"} = $ci->{total};
            $companion_stats{"${cext}_backup"}   = $c_backup;
    }
    else {
            rename( $c_backup, $cfile );
    }
    }
            else {
                $companion_stats{"${cext}_status"} = "0 records (empty)";
            }
        }
        else {
            $companion_stats{"${cext}_status"} = "already ABR v1 ($ci->{total} records)";
        }
    }

    # 6. Indeksleri insa et (gerekliyse veya eksikse)
    my $is_simple = $adb->config('use_simple') || ( $table_info && $table_info->{use_simple} ) || ( $table_info && $table_info->{id_type} && $table_info->{id_type} eq 'ascii' );
    if ( $is_simple ) {
        $self->{say} .= "  - Table '$tableid' is in simple mode, skipping index generation.\n";
    }
    elsif ( $legacy_count > 0 || $recovered_from_backup || $opts{force} ) {
        $self->{say} .= "  - Rebuilding indexes...\n";
        eval {
            ($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index( $tableid, @decoded_records );
            1;
        } or do {
            warn "  [ERROR in set_index]: $@\n";
        };
    }
    elsif ($missing_indexes) {
        $self->{say} .= "  - Rebuilding missing indexes...\n";
        eval {
            ($self->can('set_index') ? $self : do { require AmberDB::Tools::Index; AmberDB::Tools::Index->new($adb); })->set_index( $tableid, @decoded_records );
            1;
        } or do {
            warn "  [ERROR in set_index]: $@\n";
        };
    }

    # 7. Eslikci sozluk ve sayac (.unq, .cnt) dosyalarini koru ve yedekle
    my $has_unq = $adb->exist_table($tableid, 'unq');
    my $unq_backup;
    if ($has_unq) {
        my $unq_file = "$table_path.unq";
        my $u_backup_base = "$table_path-$dom_ver-$mtime_str.unq";
        $unq_backup = $u_backup_base;
        my $cnt = 1;
        while ( -e $unq_backup ) {
            $unq_backup = "$table_path-$dom_ver-$mtime_str-$cnt.unq";
            $cnt++;
        }
        require File::Copy;
        File::Copy::copy( $unq_file, $unq_backup );
    }

    my $has_cnt = $adb->exist_table($tableid, 'cnt');
    my $cnt_backup;
    if ($has_cnt) {
        my $cnt_file = "$table_path.cnt";
        my $c_backup_base = "$table_path-$dom_ver-$mtime_str.cnt";
        $cnt_backup = $c_backup_base;
        my $cnt = 1;
        while ( -e $cnt_backup ) {
            $cnt_backup = "$table_path-$dom_ver-$mtime_str-$cnt.cnt";
            $cnt++;
        }
        require File::Copy;
        File::Copy::copy( $cnt_file, $cnt_backup );
    }

    $self->{say} .= "Table '$tableid' updated to ABR v1: $total records migrated ($legacy_count converted, backup: $backup_file).\n";

    return {
        status          => ( ( $legacy_count > 0 || $companion_legacy_total > 0 || $recovered_from_backup || $opts{force} ) ? 'updated' : 'already_current' ),
        table           => $tableid,
        total           => $total,
        updated         => $legacy_count,
        already_current => $already_current,
        dominant_format => $dom_ver,
        backup_file     => $backup_file,
        del_migrated    => $companion_stats{del_migrated} // 0,
        del_backup      => $companion_stats{del_backup},
        del_status      => $companion_stats{del_status},
        aut_migrated    => $companion_stats{aut_migrated} // 0,
        aut_backup      => $companion_stats{aut_backup},
        aut_status      => $companion_stats{aut_status},
        has_unq         => $has_unq ? 1 : undef,
        unq_backup      => $unq_backup,
        has_cnt         => $has_cnt ? 1 : undef,
        cnt_backup      => $cnt_backup,
    };
}

# ---------------------------------------------------------------------
# update_all(%options)
# Iterates through all discovered tables and runs update_table on each.
# ---------------------------------------------------------------------
sub update_all {
    my ( $self, %opts ) = @_;
    my $adb = $self->{_adb} or return;

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

            $converted{$tableid} = 1;
            $self->{say} .= "  -> Rebuilt packed binary indexes (.inx, .fld, .src) for $tableid\n";
        }
    }

    return \%converted;
}

# Creates a compressed, portable .amberdb archive containing table schemas,
# native data files (.db, .del, .aut, .cnt), and integrity manifest.
# my $archive = $tools->dump( [file => 'backup.amberdb'], [tables => ['t1', 't2']] );
# ---------------------------------------------------------------------


# ---------------------------------------------------------------------
# update_version(%opts)
# Checks MetaCPAN API for latest AmberDB release and optionally installs via cpanm.
# ---------------------------------------------------------------------
sub update_version {
    my ( $self, %opts ) = @_;
    my $opt_check = $opts{check} // 0;

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

            $target_dir = $cwd;
        }
        else {
            $target_dir = $cwd;
        }
    }
    $target_dir = eval { Cwd::abs_path($target_dir) } // $target_dir;

    my $opt_force     = $opts{force} // 0;
    my $opt_check     = $opts{check} // 0;
    my $opt_no_backup = $opts{no_backup} // $opts{'no-backup'} // 0;
    my $opt_tables    = $opts{tables} // $opts{table} // '';
    my $opt_manifest  = $opts{manifest} // '';

    print "=================================================================\n";
    print " AmberDB Storage, Directory & Compatibility Migration Engine     \n";
    print "=================================================================\n";
    print "Database Directory : $target_dir\n";

    my $config_dir = File::Spec->catdir( $target_dir, "config" );
    my $ver_file   = File::Spec->catfile( $config_dir, "storage_version.json" );

lib/AmberDB/Tools/Update.pm  view on Meta::CPAN

        }
        if ( $v_curr < version->parse('5.25.0') || $opt_force ) {
            print "  - [v5.25.0] Directory migration: Rename tables/ -> table/\n";
            print "  - [v5.25.0] Format migration: Convert legacy records to ABR v5 binary pack\n";
            print "  - [v5.25.0] Index migration: Rebuild all secondary indexes (.inx, .fld, .unq, .fac, .slg, .srt)\n";
        }
        print "=================================================================\n";
        return { status => 'check', current_version => $current_storage_ver, target_version => $target_storage_ver };
    }

    # 3. Pre-migration Safety Snapshot (unless --no-backup)
    if ( !$opt_no_backup ) {
        print "Taking pre-migration safety snapshot...\n";
        my $backup_file = File::Spec->catfile( $target_dir, "backup_pre_storage_update_" . time() . ".amberdb" );
        eval {
            require AmberDB;
            my $snap_adb = AmberDB->new( path => { dbase_dir => $target_dir } );
            my $maintainer = $self->can('dump') ? $self : do {
                require AmberDB::Tools::Maintain;
                AmberDB::Tools::Maintain->new($snap_adb);
            };
            $maintainer->dump( file => $backup_file );
        };
        if ( -e $backup_file && -s $backup_file ) {
            print "  [+] Safety snapshot created: $backup_file\n";
        }
        else {
            print "  [INFO] Snapshot skipped or empty database.\n";
        }
    }

    # 4. Stage v5.21.0 Migration: Rename scheme -> schema
    if ( $v_curr < version->parse('5.21.0') ) {
        print "\n>>> [Stage v5.21.0 Migration] Scheme to Schema Directory Renaming...\n";
        $self->_inline_migrate_5_21_0($target_dir);

migrations/convert_allkeys.pl  view on Meta::CPAN

my @files = sort readdir($dh);
closedir($dh);

my %tables;
my %side_files = ( del => 0, aut => 0, cnt => 0 );

foreach my $file ( sort @files ) {
    next if $file =~ /^\./; # skip . and .. and hidden files
    $file =~ s|\Q$tables_dir\E/?||i;
    next if $file =~ /^\_/; # skip temporary/hidden files
    next if $file =~ /\s/;  # skip backup filenames with spaces

    if ( $file =~ /^([a-z0-9_]+)\.(db|inx)$/i ) {
        $tables{$1} = 1;
    }
    elsif ( $file =~ /\.del$/i ) {
        $side_files{del}++;
    }
    elsif ( $file =~ /\.aut$/i ) {
        $side_files{aut}++;
    }

migrations/update_tables.pl  view on Meta::CPAN

#!/usr/bin/perl

# bin/update_tables.pl - Automated Table Migration Utility for AmberDB
# Migrates historical tables (2003-2026 formats) to the latest ABR v1 binary format.
# Automatically creates timestamped backups with detected version tag:
#   catalog_product.db -> catalog_product-v3-2018-0315.db

use 5.016;
use strict;
use warnings;

use Getopt::Long qw(GetOptions);
use FindBin;
use lib "$FindBin::Bin/../lib";
use AmberDB;

migrations/update_tables.pl  view on Meta::CPAN

        my $err = $res->{error} // 'Unknown error';
        print "  [ERROR] FAILED! ($err)\n";
        next;
    }

    $total_processed++;

    if ($res->{status} eq 'already_current') {
        print "  [OK] ALREADY ABR v1 (" . ($res->{already_current} // 0) . " records)\n";
        if ($res->{del_migrated}) {
            print "  - Archive (.del)   : $res->{del_migrated} records migrated (Backup: $res->{del_backup})\n";
        }
        elsif (defined $res->{del_status}) {
            print "  - Archive (.del)   : $res->{del_status}\n";
        }
        if ($res->{aut_migrated}) {
            print "  - Audit Log (.aut) : $res->{aut_migrated} entries migrated (Backup: $res->{aut_backup})\n";
        }
        elsif (defined $res->{aut_status}) {
            print "  - Audit Log (.aut) : $res->{aut_status}\n";
        }
        $total_current++;
    }
    elsif ($res->{status} eq 'updated') {
        print "  [OK] MIGRATED!\n";
        print "  - Records Migrated : $res->{total} (Legacy: $res->{updated}, ABR: $res->{already_current})\n";
        print "  - Dominant Format  : $res->{dominant_format}\n";
        print "  - Backup Created   : $res->{backup_file}\n";
        if ($res->{del_migrated}) {
            print "  - Archive (.del)   : $res->{del_migrated} records migrated (Backup: $res->{del_backup})\n";
        }
        elsif (defined $res->{del_status}) {
            print "  - Archive (.del)   : $res->{del_status}\n";
        }
        if ($res->{aut_migrated}) {
            print "  - Audit Log (.aut) : $res->{aut_migrated} entries migrated (Backup: $res->{aut_backup})\n";
        }
        elsif (defined $res->{aut_status}) {
            print "  - Audit Log (.aut) : $res->{aut_status}\n";
        }
        if ($res->{has_unq}) {
            print "  - Unique (.unq)    : Preserved (Backup: $res->{unq_backup})\n";
        }
        if ($res->{has_cnt}) {
            print "  - Counter (.cnt)   : Preserved (Backup: $res->{cnt_backup})\n";
        }
        $total_updated++;
        $total_records += $res->{total};
    }
    else {
        print "$res->{status}\n";
    }
}

print "=================================================================\n";

t/amberdb_backup.t  view on Meta::CPAN

use AmberDB::Tools;

my $tmpdir = tempdir( CLEANUP => 1 );
$tmpdir =~ s{\\}{/}g;

my $adb = AmberDB->new(
    path => { dbase_dir => $tmpdir },
    cfg  => { user => "admin", language => "tr" },
);

ok( defined $adb, "AmberDB instance created for backup tests" );

# =========================================================================
# SUBTEST 1: recs_back Daily WAL CSV Stream (backup/YYYY/YYYY-MM-DD.csv)
# =========================================================================
subtest "1. Continuous Recovery Stream (recs_back -> YYYY-MM-DD.csv)" => sub {
    plan tests => 6;

    # Create a test table
    my $schema = {
        name         => 'audit_items',
        record_index => 1,
        search_block => [ 1 ],
        log_owner    => 1,

t/amberdb_backup.t  view on Meta::CPAN

    # Insert a record
    my $rid = $adb->insert_id( 'audit_items', undef, 'Keyboard', 450 );
    ok( $rid > 0, "Record inserted with ID $rid" );

    # Edit record
    $adb->modify_id( 'audit_items', $rid, 'Mechanical Keyboard', 550 );

    # Delete record
    $adb->delete_id( 'audit_items', $rid );

    # Verify backup directory structure
    my $year = $adb->{date}->{year};
    my $date_iso = "$adb->{date}->{year}-$adb->{date}->{month}-$adb->{date}->{day}";
    my $csv_file = "$tmpdir/backup/$year/$date_iso.csv";

    ok( -d "$tmpdir/backup/$year", "Year directory backup/$year created" );
    ok( -e $csv_file, "Daily CSV stream file $csv_file created" );

    # Read CSV lines
    open my $fh, "<:encoding(UTF-8)", $csv_file or die "Cannot open $csv_file: $!";
    my @lines = <$fh>;
    close $fh;

    ok( scalar(@lines) >= 3, "CSV contains at least 3 log entries (add, edit, del)" );

    # Check line format: timestamp \t user \t action \t table \t rid \t values

t/amberdb_backup.t  view on Meta::CPAN

    $adb->insert_id( 'catalog_product', undef, 'Desk Lamp', 'Furniture', 350 );
    $adb->insert_id( 'catalog_product', undef, 'Office Chair', 'Furniture', 1200 );

    # Create a unique/dictionary index (.unq) for block 2
    $adb->insert_strs( 'catalog_product', 2, [ 10, 'Electronics' ], [ 20, 'Furniture' ] );

    $adb->insert_id( 'orders', undef, 'Alice', 15350 );
    $adb->insert_id( 'orders', undef, 'Bob', 1200 );

    # Run full dump
    my $dump_file = "$tmpdir/backup/test_dump.amberdb";
    my ($outfile, $manifest) = $tools->dump( file => $dump_file );

    ok( defined $outfile, "dump() returned output path" );
    is( $outfile, $dump_file, "dump() created requested archive path" );
    ok( -e $dump_file, "Archive file exists on disk" );
    ok( -s $dump_file > 0, "Archive file is non-empty" );

    # Validate returned in-memory manifest structure
    ok( defined $manifest, "dump() returned manifest hashref" );
    is( $manifest->{format}, 'AmberDB Archive', "Manifest format is 'AmberDB Archive'" );

t/amberdb_backup.t  view on Meta::CPAN

    ok( exists $manifest->{tables}->{catalog_product}, "Manifest contains catalog_product table metadata" );
    is( $manifest->{tables}->{catalog_product}->{records}, 3, "Manifest records 3 records for catalog_product" );
};

# =========================================================================
# SUBTEST 3: Archive Package Inspection (.tar Extraction & Index Exclusion)
# =========================================================================
subtest "3. Archive Package Inspection & Integrity" => sub {
    plan tests => 8;

    my $dump_file = "$tmpdir/backup/test_dump.amberdb";
    my $tar = Archive::Tar->new();
    ok( $tar->read($dump_file), "Archive is a valid tar archive" );

    my @files = $tar->list_files();
    ok( grep { $_ eq 'manifest.json' } @files, "Archive contains manifest.json" );
    ok( grep { $_ eq 'schema/catalog.dbase' } @files, "Archive contains schema/catalog.dbase" );
    ok( grep { $_ eq 'schema/catalog_product.table' } @files, "Archive contains schema/catalog_product.table" );
    ok( grep { $_ eq 'table/catalog_product.dumpdb' } @files, "Archive contains table/catalog_product.dumpdb" );
    ok( grep { $_ eq 'table/orders.dumpdb' } @files, "Archive contains table/orders.dumpdb" );

t/amberdb_backup.t  view on Meta::CPAN

    my $sha = $manifest->{tables}->{catalog_product}->{sha256}->{'table/catalog_product.dumpdb'};
    ok( defined $sha && length($sha) == 64, "Manifest contains valid 64-character SHA-256 hash" );
};

# =========================================================================
# SUBTEST 4: Tools->restore() and Deterministic Index Rebuilding
# =========================================================================
subtest "4. Database Restore and Index Reconstruction" => sub {
    plan tests => 17;

    my $dump_file = "$tmpdir/backup/test_dump.amberdb";

    # 4.1 Safety check: Restore into non-empty database without force should fail
    my $tools = AmberDB::Tools->new($adb);
    my $blocked_res = $tools->restore( file => $dump_file, force => 0 );
    ok( !defined $blocked_res, "restore() without force on non-empty database returns undef" );

    # 4.2 Create a completely clean staging database directory
    my $stagedir = tempdir( CLEANUP => 1 );
    $stagedir =~ s{\\}{/}g;

t/amberdb_encapsulation.t  view on Meta::CPAN

    plan tests => 6;

    # Getter
    is( $adb->path('dbase_dir'), $tmpdir, "path('dbase_dir') returns expected directory" );

    # Setter via key-value
    $adb->path( custom_path => "/opt/data" );
    is( $adb->path('custom_path'), "/opt/data", "path(custom_path => ...) sets path" );

    # Setter via hashref
    $adb->path({ backup_dir => "$tmpdir/backup" });
    is( $adb->path('backup_dir'), "$tmpdir/backup", "path({ backup_dir => ... }) sets path" );

    # Bulk copy
    my $paths_copy = $adb->path();
    ok( ref($paths_copy) eq 'HASH', "path() returns HASH ref" );
    is( $paths_copy->{dbase_dir}, $tmpdir, "path copy contains dbase_dir" );

    # Mutation protection
    $paths_copy->{dbase_dir} = "/compromised/path";
    is( $adb->path('dbase_dir'), $tmpdir, "Internal path is NOT affected by mutating copy" );
};

t/amberdb_simple_mode.t  view on Meta::CPAN


    # 2. Modify record (edit action)
    $adb->modify_id( 'sessions', $sid, 'Closed', '192.168.1.50' );

    # 3. Delete record (del action)
    $adb->delete_id( 'sessions', $sid );

    my $date_iso = "$adb->{date}->{year}-$adb->{date}->{month}-$adb->{date}->{day}";
    my $csv_file = "$db_dir/$date_iso.csv";

    ok( -e $csv_file, "Daily CSV backup file $csv_file created directly in same dbase_dir in simple mode" );
    ok( !-d "$db_dir/backup", "No separate backup directory created in simple mode" );

    open my $fh, "<:encoding(UTF-8)", $csv_file or die "Cannot open $csv_file: $!";
    my @lines = <$fh>;
    close $fh;

    is( scalar(@lines), 3, "CSV contains exactly 3 entries (add, edit, del) in simple mode" );

    # Line 1: add
    chomp $lines[0];
    my @cols_add = split /\t/, $lines[0];

t/amberdb_simple_mode.t  view on Meta::CPAN

    my @cols_edit = split /\t/, $lines[1];
    is( $cols_edit[2], 'edit', "Col 2 is action 'edit'" );
    is( $cols_edit[4], $sid,   "Col 4 is custom string ID" );

    # Line 3: del
    chomp $lines[2];
    my @cols_del = split /\t/, $lines[2];
    is( $cols_del[2], 'del', "Col 2 is action 'del'" );
    is( $cols_del[4], $sid,  "Col 4 is custom string ID" );

    # 4. Verify no_backup flag disables logging
    my $db_dir_noback = "$tmp_dir/simple_db8_noback";
    mkdir($db_dir_noback);
    my $adb_noback = AmberDB->new(
        path => { dbase_dir => $db_dir_noback },
        cfg  => { use_simple => 1, no_backup => 1 },
    );
    $adb_noback->insert_id( 'tbl', 'id1', 'val1' );
    my $noback_csv = "$db_dir_noback/$date_iso.csv";
    ok( !-e $noback_csv, "No backup CSV created when no_backup => 1" );
};

# ============================================================
# 9. Simple Mode Key Sanitization and Reference Rejection
# ============================================================
subtest '9. Simple Mode Key Sanitization and Reference Rejection' => sub {
    my $db_dir = "$tmp_dir/simple_db9";
    mkdir($db_dir);
    my $adb = $tools->db_simple($db_dir);

t/amberdb_update_table.t  view on Meta::CPAN

    $dbh->put("2", "Product 2005\tred\\Tblue");
    $dbh->put("3", "Product 2021<TAB0>opt1<TAB1>opt2");
    $dbh->put("4", "Product 2026\tARRAY:t1|t2\tHASH:k=v");
    $adb->table_close($table_file);

    # Run update_table
    my $res = $tools->update_table($tbl);
    is( $res->{status}, 'updated', 'update_table returned status updated' );
    is( $res->{total}, 4, 'Total records processed is 4' );
    is( $res->{updated}, 4, '4 legacy records were converted' );
    ok( -e $res->{backup_file}, "Backup file was created: $res->{backup_file}" );
    like( $res->{backup_file}, qr/catalog_products-v\d-\d{4}-\d{4}\.db/, 'Backup file matches -v<ver>-<date>.db naming pattern' );

    # Verify table is now 100% ABR v1
    $adb->table_read($table_file);
    my %migrated;
    $adb->recs_scan($table_file, sub {
        my ($k, $v) = @_;
        $migrated{$k} = $v;
    });
    $adb->table_close($table_file);

t/amberdb_update_table.t  view on Meta::CPAN

    # Write .del archive in 2005 \T format
    $adb->table_write($del_file);
    my $ddbh = $adb->{_db}->{$del_file};
    $ddbh->put("999", "Order Cancelled\titem1\\Titem2\t0");
    $adb->table_close($del_file);

    # Run update_table
    my $res = $tools->update_table($tbl);
    is( $res->{status}, 'updated', 'Orders table updated' );
    is( $res->{del_migrated}, 1, '1 record in .del archive migrated' );
    ok( -e $res->{del_backup}, "Backup file for .del created: $res->{del_backup}" );

    # Verify .del file is now ABR v1
    $adb->table_read($del_file);
    my %del_migrated;
    $adb->recs_scan($del_file, sub {
        my ($k, $v) = @_;
        $del_migrated{$k} = $v;
    });
    $adb->table_close($del_file);

t/amberdb_update_table.t  view on Meta::CPAN

    is( $adb->exist_table($tbl, 'aut'), 1, 'exist_table detects .aut' );
    is( $adb->exist_table($tbl, 'cnt'), 1, 'exist_table detects .cnt' );
    is( $adb->exist_table($tbl, 'slg'), 0, 'exist_table returns 0 for non-existent .slg' );

    # Run update_table
    my $res = $tools->update_table($tbl);
    is( $res->{status}, 'updated', 'Products table updated successfully' );

    # Verify .unq was PRESERVED and NOT unlinked
    is( $res->{has_unq}, 1, 'Report acknowledges .unq was found' );
    ok( -e $res->{unq_backup}, "Backup copy of .unq created: $res->{unq_backup}" );
    is( $adb->exist_table($tbl, 'unq'), 1, 'Live .unq file is preserved and still exists!' );

    # Verify data inside live .unq was NOT deleted or corrupted
    $adb->table_read($unq_file);
    my $unq_val = $adb->recs_get($unq_file, "title:s:Wireless Keyboard");
    $adb->table_close($unq_file);
    is( $unq_val->{"title:s:Wireless Keyboard"}, 42, '.unq dictionary mapping retained completely' );

    # Verify .aut was migrated and backed up
    is( $res->{aut_migrated}, 1, '.aut audit entry migrated' );
    ok( -e $res->{aut_backup}, "Backup of .aut created: $res->{aut_backup}" );

    # Verify .cnt was preserved and backed up
    is( $res->{has_cnt}, 1, '.cnt counter preserved' );
};

done_testing();



( run in 3.210 seconds using v1.01-cache-2.11-cpan-99fa8243a42 )