AmberDB
view release on metacpan or search on metacpan
Revision history for AmberDB
5.24.0 2026-09-05
- [BINARY RECORD SERIALIZATION ARCHITECTURE (ABR v5)] Native Pure Perl Binary Serialization:
* Introduced high-performance native pure Perl binary record serialization format (ABR v5 / Format 5) replacing legacy delimiter and regex text serialization (db_encode / db_decode), aligning format versioning with historical eras (v1: 2003...
* Zero CPAN Dependencies: Built strictly on core built-in Perl primitives (pack, unpack, substr, vec), completely eliminating version brittleness and security vulnerabilities associated with external serializers like Storable.
* Magic Header Architecture: Prefixes binary records with a 5-byte magic sequence (\x00ABR\x05); null-byte prefix guarantees zero collision with legacy plain text or data strings.
* Schema Type Coverage: Transparently maps all 9 AmberDB schema types into 1-byte typed nodes: UNDEF (0x00), SCALAR_RAW (0x01), SCALAR_UTF8 (0x02, via lossless utf8::encode/decode), ARRAY (0x03, 16-bit Big-Endian count), and HASH (0x04, 16-...
* Nested Data Structures: Fully supports arbitrarily nested arrays, hashes, and repeat blocks with strict recursion depth guarding ($depth <= 32) to prevent stack overflow or circular reference hangs.
* Transparent Legacy Fallback: db_decode automatically falls back to _db_decode_legacy for non-ABR records, allowing mixed-version legacy tables to operate without downtime.
* Benchmarks: Achieves ~130,000 encodes/sec (+150% faster) and ~69,000 decodes/sec (+64% faster) on flat records; achieves ~4,300 decodes/sec (+35% faster) on complex deeply nested multi-level records.
- [BINARY INDEX ARCHITECTURE REFACTORING] Complete Migration of All Indexes to 8-Byte Packed Binary Buffers:
* Migrated all secondary index subsystems (.inx, .fld, .src, .fac, .slg, and Tier B Junk .jinx, .jfld, .jsrc) to pure 8-byte fixed-width packed binary buffers (pack "Q>", unpack "(Q>)*").
* Core Binary Primitives in AmberDB::Base: Implemented bin_add, bin_punch, bin_sort, bin_find, and bin_count operating directly on raw byte buffers via substr() and memory-aligned index(), achieving C-level execution speed.
* High-Level Cleanup: Completely eradicated high-level Perl array/hash manipulations (array_nodup, array_punch) from the core engine indexing path.
* Consolidated Pre-Sorted Indexing: Standalone .srt files are formally deprecated and eliminated; sort indexes are directly maintained inside .inx.
* Direct 64-bit Uint Indexing: Non-foreign key numeric fields in .fld bypass synthetic dictionary ID generation, indexing pure 64-bit unsigned integers directly into binary keys.
* Added comprehensive test suite t/amberdb_update_table.t covering multi-era format decoding, update_table migration, backup naming, .del/.aut/.cnt/.unq handling, and update_all batch discovery.
* Updated MANIFEST to include bin/update_tables.pl and new test suites.
* All 47 test files (440 assertions) passing with 100% success rate.
5.23.2 2026-09-03
- [LOCALE ENGINE & MULTILINGUAL ARCHITECTURE] 10th Language - Global Base (gb) and Default Locale:
* Introduced Global Base (gb) as the 10th supported language and new universal default/fallback locale (replacing en).
* Implemented comprehensive multilingual Latin character preservation in alphabet_chars across European, Turkish, Nordic, French, German, Spanish, and Slavic-Latin alphabets.
* Added cross-lingual, accent-tolerant search regex mapping (regex_map) matching accented and unaccented variations (e.g. cafe matches café, munchen matches münchen, seker matches şeker).
* Implemented canonical accent folding in accent_map for high-recall inverted search indexing (.src).
* Added lossless Unicode ligature conversion in ascii_map (ß->ss, æ->ae, œ->oe, ı->i, ø->o, ł->l, đ->d, ð->d, þ->th, ə->e) for clean URL slug and ASCII ID generation.
* Configured international English numbering, date formatting, and ISO standard decimal/group separators.
* Added language aliases: 'gb', 'global', 'gl', 'universal', 'uni', 'gb_base'.
* Added dedicated test coverage in t/amberdb-locale_09_gb.t.
- [TRANSACTION ARCHITECTURE & API REFINEMENT] Pure File-Path Error Model and Operational Rollback:
* Refactored transact_error($file_path, $message) to exclusively accept physical file paths; eliminated artificial "transaction" and "system" string contexts.
* Simplified table identification via single exact regex /([^\/\\:]+)\.$db_ext$/: directly extracts table ID and inspects schema no_transact attribute; non-db extensions (.inx, .src, .fld, .fac, .slg, .aut, .del, .txn) never trigger rollbac...
* Added immediate early return in transact_error if $file_path is undefined or empty.
* Established strict API role separation: application code directly invokes transact_rollback() for business logic cancellations (insufficient stock, credit limits, validation aborts) and unexpected eval exceptions; transact_error is reserv...
* Unified legacy is_index and is_no_transact flags into single no_rollback attribute.
* Restructured insert_id to strictly validate mandatory $tableid at entry prior to resolving table paths, with proper ref guard for $rid.
docs/EN.About_AmberDB.md view on Meta::CPAN
---
## Why AmberDB?
AmberDB brings together diverse capabilities within a single Perl database engine:
* **No External Database Server:** Embedded directly into the application as an in-process object.
* **Array-Based Records:** Database records are Arrays (lists) supporting JSON-like nested structures.
* **Full CRUD Operations:** Fast, direct insert, read, update, and delete methods.
* **Flexible Indexing:** Operates seamlessly both with and without indexes. Requires no separate setup for indexing.
* **Full-Text Search:** High-performance search (both indexed and unindexed).
* **Sorting & Versatile Queries:** Multi-field sorting, range queries, and regex filtering.
* **Nested & Repeating Blocks:** Automatic management of relational and repeating data blocks.
* **Multi-Table Relational Transactions:** ACID-compliant transaction management with Strict 2-Phase Locking (Strict 2PL).
* **Faceted Filtering:** Dynamic faceted filtering generation, just like on modern e-commerce sites.
* **RAM-to-Disk Tiering:** RAM buffer caching layer similar to Redis.
* **High-Throughput Batch Processing:** Dedicated bulk ingestion and index-merge pipeline.
* **Audit Logging:** Built-in logging of user operations on each record.
* **Soft Deletes:** Table-definition-specific soft delete (stores deleted records in an archive tier).
* **Automatic SEO Slugs:** Automatic URL slug generation for catalog tables.
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
my ($total, @sorted_alpha) = $adb->read_all("catalog_product", 0, 20, sort => { blk => 4, reverse => 1 });
my ($total, @highest_price)= $adb->read_all("catalog_product", 0, 10, sort => 10);
my ($total, @lowest_price) = $adb->read_all("catalog_product", 0, 10, sort => -10);
# 3.4 Paginated and tiered (Active + Junk)
my ($total, @tiered_page) = $adb->read_all("catalog_product", 0, 20, jnktype => 'AB');
```
### 4.2 `field_fetch` — Inverted Match Index (.fld) and Multi-Value Querying
Fields defined in `match_block` are retrieved via inverted match indexes (`.fld`) with O(1) average lookup time per indexed key (when querying multiple values, cost scales with the number of keys). Even if a record stores multiple comma-separated IDs...
```perl
# 1. Fetch all products where Category ID (Block 1) matches "5"
my @products = $adb->field_fetch("catalog_product", 1, "5");
# 2. Fetch all products by Author ID (Block 3) "9" (Matches even if record has "7,9")
my @author_prods = $adb->field_fetch("catalog_product", 3, "9");
# 3. Paginated & sorted: Category 5 products sorted by Price (Block 10) ascending
my ($count, @sorted_prods) = $adb->field_fetch(
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
sort => { blk => 10, reverse => 1 } # Sort by price ascending
);
# 3. Retrieve only matching record IDs (keys_only)
my ($count, @id_list) = $adb->search_table("catalog_product", "sony", 0, 50, keys_only => 1);
my @all_ids = $adb->search_table("catalog_product", "sony", keys_only => 1);
```
#### Key Highlights of AmberDB Search Normalization:
- **Apostrophe / Suffix Handling:** In records containing `"Türkiye'nin"`, queries for `"Türkiye"`, `"Türkiye'nin"`, and `"Türkiyenin"` all match. Suffixes following apostrophes (`"nin"`, `"da"`, `"in"`) are stripped as stop-words.
- **Final Consonant Devoicing (Phonetic Assimilation):** Automatic phonetic mapping for word-final consonants (`b$ => p`, `d$ => t`, `g$ => k`), seamlessly matching queries like `"tevhid"` $\leftrightarrow$ `"tevhit"`, `"gazab"` $\leftrightarrow$ `"g...
- **Circumflex Vowels:** Accented vowels (`â, î, û`) match standard vowels: `"kârın"` $\leftrightarrow$ `"karın"`, `"ÂLÎM"` $\leftrightarrow$ `"alim"`.
- **Character & ASCII Equivalence:** Full case-insensitive and Turkish/ASCII folding (`"ığdır"` $\leftrightarrow$ `"IĞDIR"` $\leftrightarrow$ `"igdir"`, `"ÇARŞI"` $\leftrightarrow$ `"çarşı"` $\leftrightarrow$ `"carsi"`, `"ÇÖPÇÜ"` $\leftr...
### 4.5 `read_list` — Reading Specific IDs in Specified Sequence
`read_list` is AmberDB's high-throughput batch record resolution engine. It plays an essential role both in the engine's internal query pipeline and in developer application code:
#### 1. Internal Engine Pipeline:
All high-level listing and querying methods in AmberDB (`read_all`, `field_fetch`, `search_table`, `field_filter`, etc.) operate in two decoupled stages:
1. **Index Filtering Stage:** The query method first reads lightweight record keys (`@ids`) from inverted index files (`.inx`, `.fld`, `.src`, `.srt`), evaluating Boolean logic (AND/OR), sorting, and pagination slicing (`recs_cutting`).
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
```perl
$adb->insert_id( 'sessions', 'user@example.com', 'Active', 'Chrome', time() );
my @sess = $adb->read_id( 'sessions', 'user@example.com' );
```
---
### 5.3 Data Operations (CRUD & Bulk)
All standard CRUD and bulk methods operate seamlessly in Simple Mode:
```perl
# Single Insert, Read, Modify, Delete
$adb->insert_id( 'orders', 'order_101', 'Pending', '150.00' );
my @order = $adb->read_id( 'orders', 'order_101' );
$adb->modify_id( 'orders', 'order_101', 'Completed', '175.50' );
$adb->delete_id( 'orders', 'order_101' );
my $exists = $adb->exist_id( 'orders', 'order_101' );
# Bulk Operations (Bulk CRUD)
docs/EN.AmberDB_User-Guide.md view on Meta::CPAN
#### C. Back-Office Admin & Reports (Archived Items Only - Mode `B`)
Inspect discontinued, out-of-stock, or passive catalog items:
```perl
# List all archived/junk product IDs:
my @archived_ids = $adb->read_all("catalog_product", jnktype => "B", keys_only => 1);
```
#### D. Order & Invoice Processing (Direct ID Access)
Past orders access product details seamlessly regardless of whether the item is active or archived:
```perl
# Fetch product details directly by ID (Works instantly for both active and archived products):
my @product = $adb->read_id("catalog_product", $old_product_id);
```
### 11.4 Automatic State Migration
When updating a product, AmberDB evaluates the schema rules in real time:
* Setting `sales_status` to `0` or disabling a vendor automatically **demotes the product from storefront to archive**.
* Restocking the item and setting status back to `1` automatically **restores the product to the active storefront**.
docs/index.md view on Meta::CPAN
### 2. 64-Bit Big-Endian Binary Indexing
Primary and secondary indexes use fixed 8-byte packed Big-Endian unsigned integer buffers (`Q*`). This guarantees $O(1)$ binary slicing, zero string-unpacking heuristics, and sub-millisecond pagination even across datasets scaling into millions of ro...
### 3. ACID Transactions with Strict 2PL
Full multi-table transaction support with a disk-backed undo journal (`.txn`) and Strict Two-Phase Locking (Strict 2PL). Abnormal terminations trigger automatic LIFO rollbacks upon recovery.
### 4. High-Throughput Batch Ingestion
Bulk ETL methods (`insert_list`, `modify_list`, `delete_list`) open master tables once and merge indexes in a single pass, delivering 50x–100x higher throughput compared to single-record loops.
### 5. Multi-Tier Junk & Archiving
Active records (`.db`) are seamlessly segregated from historical or archived rows (`.jnk`), supporting unified single-pass queries (`jnktype => 'A' | 'B' | 'AB' | 'BA'`).
### 6. Faceted Category Filter Engine
Built-in columnar facet indexing (`.fac`) with bitwise set intersections and string dictionaries (`.str`) enables instant e-commerce filtering menus without external search appliances.
---
## 📊 Feature Comparison
| Capability | AmberDB | SQLite | Traditional RDBMS (PostgreSQL/MySQL) |
| :--- | :--- | :--- | :--- |
lib/AmberDB/Locale/Lang/gb.pm view on Meta::CPAN
# cross-border datasets combining English, Turkish, German,
# French, Spanish, Italian, Scandinavian, and Slavic-Latin text.
# 2. Comprehensive Alphabet: alphabet_chars includes all European,
# Nordic, and Turkish extended Latin letters so no valid characters
# are stripped during text sanitization.
# 3. Permissive Search Regex: regex_map expands characters across their
# accented and unaccented forms (e.g. searching 'cafe' matches 'café',
# 'munchen' matches 'münchen', 'seker' matches 'şeker').
# 4. Canonical Accent Folding: accent_map flattens accented letters
# to base Latin forms for high-recall inverted search indexing (.src).
# 5. Lossless ASCII Transliteration: ascii_map cleanly converts
# non-decomposable Unicode ligatures (ß->ss, æ->ae, œ->oe, ø->o,
# ı->i, ł->l, ð->d, þ->th, ə->e) for URL slugs and ASCII IDs.
# 6. International Formatting: Default numbers, dates, and currency
# follow ISO and international English conventions (USD/cent, . decimal).
# -------------------------------------------------------
sub data {
return {
# ---------------------------------------------------------
# Casing special-cases
t/amberdb-locale_09_gb.t view on Meta::CPAN
is( $loc->normalize("Straße"), "Strasse", 'normalize: Straße -> Strasse' );
# Turkish
is( $loc->normalize("ışık"), "isik", 'normalize: ışık -> isik' );
# Nordic
is( $loc->normalize("smørrebrød"), "smorrebrod", 'normalize: smørrebrød -> smorrebrod' );
};
# ============================================================
# 4. Lossless ASCII Transliteration (to_ascii / slug)
# ============================================================
subtest '4. Lossless ASCII Transliteration (to_ascii)' => sub {
plan tests => 8;
my $loc = AmberDB::Locale->new('gb');
is( $loc->to_ascii("Straße"), "Strasse", 'to_ascii: ß -> ss' );
is( $loc->to_ascii("Straße", 1), "strasse", 'to_ascii slug: Straße -> strasse' );
is( $loc->to_ascii("bærum"), "baerum", 'to_ascii: æ -> ae' );
is( $loc->to_ascii("cœur"), "coeur", 'to_ascii: œ -> oe' );
is( $loc->to_ascii("Kraków"), "Krakow", 'to_ascii: ó -> o' );
is( $loc->to_ascii("Åódź"), "Lodz", 'to_ascii: Å/ó/ź -> Lodz' );
( run in 1.255 second using v1.01-cache-2.11-cpan-aadc1410aed )