App-FileSync
view release on metacpan or search on metacpan
website/html/limitations.html view on Meta::CPAN
<div class="wrap header-inner">
<a class="brand" href="index.html">
<span class="brand-name">App::FileSync</span>
<span class="brand-tag">file synchronisation without a daemon</span>
</a>
<nav class="site-nav" aria-label="Main">
<a href="index.html">Overview</a>
<a href="quickstart.html">Quick start</a>
<a href="install.html">Install</a>
<a href="commands.html">Commands</a>
<a href="settings.html">Settings</a>
<a href="how-it-works.html">How it works</a>
<a href="limitations.html" aria-current="page">Limitations</a>
<a href="changelog.html">Changes</a>
</nav>
</div>
</header>
<main id="main" class="wrap">
<h1>Limitations</h1>
<p class="lede">These are the design weaknesses to weigh before relying on the
tool. None of them loses data silently; most cost time or space.</p>
<h2>Topology and scale</h2>
<ul>
<li><strong>One master.</strong> The topology is a star around one server.
The replica is a pull-only mirror, not a failover: if the server is lost,
clients stop until an administrator points them at a rebuilt master.</li>
<li><strong>Every run exchanges the whole manifest.</strong> The client sends
one entry per file and the server answers with the whole plan as a single
JSON line, so a run costs O(files) in time, memory and bandwidth even when
nothing changed. A tree of a million files means a plan message of the order
of a hundred megabytes. A change journal or Merkle summary would remove
that, at the cost of the "everything is inspectable with cat" property.</li>
<li><strong>No line-length cap on the wire.</strong> Because of the previous
point, a control message cannot be bounded without breaking large trees.
The peer is authenticated by ssh and pinned by the forced command, so this is
a resource risk from a misbehaving client, not from the network.</li>
<li><strong>Lock-step protocol.</strong> One request is in flight at a time
and each 256 KiB chunk waits for its acknowledgement, so throughput is
bound by the round-trip time on a high-latency link.</li>
<li><strong>Large trees cost memory.</strong> The plan holds every path in
memory three times over — the union of the three manifests — and
the server holds the whole client manifest and the plan. That is the price of
a stateless server.</li>
<li><strong>Every run stats every file.</strong> There is no change
notification. On a very large tree the walk, not the transfer, dominates a
quiet run.</li>
</ul>
<h2>Consistency</h2>
<ul>
<li><strong>Clock-dependent tie-breaks.</strong> When both sides changed a
file, the newer mtime wins (the master wins ties within two seconds). A
client with a wrong clock picks the wrong winner; the loser is still kept as
a conflict copy.</li>
<li><strong>The cache trusts size and mtime.</strong> A tool that rewrites a
file and restores its mtime — some backup restorers do — goes
unnoticed until a <code>verify --deep</code>.</li>
<li><strong>One scan per session.</strong> The plan is computed once, from a
scan taken while the lock is held. Anything other than filesync writing into
the master during a session is not seen until the next session, and a file
it replaces may be overwritten.</li>
<li><strong>Pulls are verified against the plan.</strong> If the master
changes during the session (only possible if something other than filesync
writes to it), the pull fails with a hash mismatch and the next sync
retries.</li>
<li><strong>Uploads are verified only at the end.</strong> A client that
sends a wrong file of the announced size wastes the transfer before it is
refused.</li>
</ul>
<h2>Durability</h2>
<ul>
<li><strong>No fsync.</strong> Renames are atomic between processes but not
durable across a power cut; the new name can point at a file whose data
never reached the disk. Adding <code>IO::Handle::sync</code> before each
rename would make every upload pay a disk flush; the trade-off has not been
measured.</li>
<li><strong>No extended metadata.</strong> <code>File::Copy::copy</code> does
not copy extended attributes, ACLs or alternate data streams.</li>
<li><strong>Snapshots share the disk.</strong> They live inside the replica
root, on the same filesystem: they guard against bad syncs, not against
losing that disk.</li>
</ul>
<h2>Deltas</h2>
<ul>
<li><strong>Pure Perl.</strong> The rolling loop runs at a few MB/s, which is
why only 8 MiB after the matched prefix is scanned. An edit near the end
of a large file whose earlier blocks also shifted is sent mostly
literally.</li>
<li><strong>Block-aligned moves only.</strong> A block that moved by a whole
number of blocks is found; a block that occurs twice in the receiver's file
is matched to its first occurrence.</li>
<li><strong>32-bit offsets.</strong> A delta cannot refer beyond 4 GiB
into the receiver's file; larger files fall back to whole-file transfer of
the remainder as literals.</li>
</ul>
<h2>What is not synced</h2>
<ul>
<li>Symlinks, devices, sockets, extended attributes, ACLs and ownership.
Permission bits are carried only between POSIX systems.</li>
<li><strong>Unicode normalisation is not reconciled</strong>, so the same
name in NFC (Linux) and NFD (older macOS) is two different files. A
case-insensitive client cannot hold two master paths that differ only in
case.</li>
<li><strong>A rename is a deletion plus a creation</strong>, so a renamed
large file is transferred again rather than moved.</li>
<li><strong>Ignore matching is literal and linear.</strong> Character
classes (<code>[abc]</code>) and backslash escapes are taken literally; a
trailing <code>/</code> is dropped, so <code>build/</code> matches
<code>build</code> at any depth, and as the matcher has no type information
it matches a file of that name too. Combining patterns into one alternation
would be faster but would lose "later patterns win".</li>
</ul>
<h2>Recovery and administration</h2>
<ul>
<li><strong>Trash is per repo and per client.</strong> Local trash written by
releases before 0.001.1 sits in one shared directory. It is still searched
by <code>restore</code> and pruned with the default 30 days, but a
<code>restore</code> can then offer another repo's file of the same relative
path.</li>
<li><strong>Same-day trash collisions are numeric.</strong> A real file whose
name already ends in <code>.N</code> (<code>backup.1</code>) deleted the same
day as <code>backup</code> is listed as a version of <code>backup</code>
too. Pruning works in whole UTC days, so retention is accurate to a day.</li>
<li><strong>Trash cannot cross filesystems by rename.</strong> Files are
copied and removed; directories must rename.</li>
<li><strong>Admin methods mostly take no lock.</strong> Only
<code>reindex</code> does; <code>restore</code> into the master rebuilds the
index by walking the tree and does not take the repo lock, so a session
running at the same moment can save an index that omits the restored file
until its next sync.</li>
<li><strong>The server journal is read whole.</strong> <code>filesync-admin
log</code> loads every monthly journal into memory before filtering and
applying the limit. A repo with years of history pays for all of it on every
call.</li>
<li><strong>A shallow verify trusts size and mtime</strong>, so it cannot see
a change that kept both; use <code>--deep</code> or <code>--sample</code>.
The report says what differs, not which side is right.</li>
<li><strong>Snapshot listing is slow.</strong> <code>snapshot list</code>
walks every snapshot to count it.</li>
</ul>
<h2>Costs and implementation notes</h2>
<ul>
<li><strong>Encapsulation has a price.</strong> Private methods use
Sub::Private in <code>enforce</code> mode, about 1.4 microseconds per call
(measured on 5.40). Helpers that run per file or per JSON value — the
merge comparisons, the JSON byte downgrade, the rolling checksum — are
therefore lexical code references: private by scope, with no wrapper.</li>
<li><strong>Validation has a price.</strong> Params::Validate::Strict costs
about 11 microseconds per call, so it guards the public entry points that run
a few times per command. Functions that run per path or per protocol frame
check their arguments by hand.</li>
<li><strong>English only.</strong> The catalogue mechanism supports other
languages, plural forms and gender, but only the English catalogue ships.
The plural rules cover English, German, French, Chinese, Japanese and Korean;
other languages use the English rule until one is added. Operating-system
reasons (<code>$!</code>) come from the C library and follow the process
locale, not the catalogue language.</li>
<li><strong>Option parsing is Getopt::Long's.</strong> Its diagnostics for an
unknown option are printed in English before the catalogued usage line.
<code>--json</code> output is not localised, because field names and values
are data.</li>
<li><strong><code>filesync-admin</code> reads <code>--root</code> only before
the command name.</strong></li>
<li><strong>Two configuration mechanisms.</strong> The client's repo list is
JSON written by <code>filesync init</code> and must round-trip exactly;
process settings go through Object::Configure. Two mechanisms is one too
many, but the layered merge does not promise an exact round trip.</li>
</ul>
</main>
<footer class="site-footer">
<div class="wrap">
<p>App::FileSync 0.002.0 · Copyright 2026 Nigel Horne · GPL-2.0</p>
<p><a href="https://metacpan.org/dist/App-FileSync">CPAN</a> ·
<a href="https://github.com/nigelhorne/App-FileSync">GitHub</a> ·
<a href="https://github.com/nigelhorne/App-FileSync/issues">Issues</a> ·
<a href="changelog.html">Changes</a></p>
</div>
</footer>
</body>
( run in 2.274 seconds using v1.01-cache-2.11-cpan-99fa8243a42 )