pastilhasandClaude Opus 5
01652f3480
Moving a file into a folder made the sync look at a new path, find nothing, and convert it again — minutes of work to produce bytes that already existed a directory away — while the old folder stayed behind forever, because nothing ever deleted one. A move preserves size and mtime, and both are already in the manifest and in the stat. So the sync indexes what is already derived on that pair, and a file that has moved has its ladder RENAMED across. An ambiguous match is refused rather than guessed. Two originals can share a size and an mtime — a camera dump can write a whole card in one second — and rebuilding is merely slow where attaching the wrong photographs to a name is wrong. Orphans are removed afterwards, and only on a WHOLE-TREE sync. A scoped run knows about one folder, so everything outside it looks abandoned; deleting on that basis would take out the library the first time somebody re-synced a subfolder. Safe otherwise because derived/ is reproducible by definition, which is what its name has meant since these two folders stopped being called `user` and `treated`. Both are counted, recorded in the ledger and shown only when they happened, so an ordinary run stays one short line. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
103 lines
4.0 KiB
TypeScript
103 lines
4.0 KiB
TypeScript
import { appendFile, readFile } from 'node:fs/promises';
|
|
import { join } from 'node:path';
|
|
import { mediaPath } from './media';
|
|
import type { SyncError } from './sync';
|
|
|
|
/**
|
|
* What has been synced, and when.
|
|
*
|
|
* ── The hole this fills ──
|
|
*
|
|
* A run's state lives in memory, so a restart erases it — and with it any record that a file FAILED.
|
|
* That is the part the manifests cannot cover: a derived folder records the size, mtime and variants it
|
|
* was built from, so provenance is already answerable precisely, but a file that did not build has no
|
|
* folder and leaves no trace at all. With the watcher running, a failure at three in the morning would
|
|
* simply never be discovered.
|
|
*
|
|
* So the ledger's job is RUNS AND FAILURES, not provenance. The manifests keep the latter and are
|
|
* better at it.
|
|
*
|
|
* ── A file, not a table ──
|
|
*
|
|
* `db/schema.ts` says the connection is the only thing this plugin stores, and gives the reason: the
|
|
* installer drops a plugin's tables when "delete its data" is ticked, so the strongest thing that can
|
|
* be said about this plugin is that no operation on it can destroy the client's work. A ledger is
|
|
* losable — re-run and it rebuilds — so it does not need that protection, and a table would be a
|
|
* schema, a migration and a third thing to keep in step for something a line of JSON answers.
|
|
*
|
|
* It also matches the design already in play next door: the filesystem is the record.
|
|
*
|
|
* ── Beside `derived/`, not inside it ──
|
|
*
|
|
* Wiping `derived/` is a normal thing to do and must not erase the history of what happened before it —
|
|
* that history is exactly the comparison this exists for.
|
|
*
|
|
* ── Append-only, and why that is safe ──
|
|
*
|
|
* One line per run, written with a single `appendFile` on an O_APPEND handle. Writes under a page are
|
|
* atomic on Linux, and these are a couple of hundred bytes, so two writers cannot interleave a line.
|
|
* Today there is only ever one run at a time; that will stop being true when the watcher lands, which
|
|
* is why it is worth being true by construction rather than by luck.
|
|
*/
|
|
|
|
const FILE = '.sync-log.jsonl';
|
|
|
|
export type RunTrigger = 'manual' | 'watcher';
|
|
|
|
export type Run = {
|
|
startedAt: string;
|
|
finishedAt: string;
|
|
trigger: RunTrigger;
|
|
/** The subtree covered, relative to `originals/`. Empty means all of it. */
|
|
scope: string;
|
|
/** Convertible files the walk found. */
|
|
total: number;
|
|
built: number;
|
|
skipped: number;
|
|
/** Renamed into place after a move, rather than rebuilt. */
|
|
moved: number;
|
|
/** Removed because the original is gone. */
|
|
pruned: number;
|
|
errors: SyncError[];
|
|
/** Set when the run stopped for a reason that was not about any single file. */
|
|
fatal: string | null;
|
|
};
|
|
|
|
const ledgerPath = async (email: string): Promise<string> => join(await mediaPath(email), FILE);
|
|
|
|
export async function appendRun(email: string, run: Run): Promise<void> {
|
|
try {
|
|
await appendFile(await ledgerPath(email), `${JSON.stringify(run)}\n`, 'utf8');
|
|
} catch {
|
|
/*
|
|
Swallowed on purpose. Failing to RECORD a sync must never fail the sync — the files are the
|
|
product and this is the note about them. The most likely cause is `media/` not existing, which
|
|
the panel already reports far more usefully than an exception thrown from here would.
|
|
*/
|
|
}
|
|
}
|
|
|
|
/**
|
|
* The most recent runs, newest first.
|
|
*
|
|
* Read whole and sliced. The file is a couple of hundred bytes a line and would take years of a
|
|
* watcher firing on every upload to become worth streaming; when it does, that is the moment to
|
|
* change this, and not before.
|
|
*/
|
|
export async function readRuns(email: string, limit = 20): Promise<Run[]> {
|
|
const raw = await readFile(await ledgerPath(email), 'utf8').catch(() => '');
|
|
|
|
const runs: Run[] = [];
|
|
for (const line of raw.split('\n')) {
|
|
if (!line.trim()) continue;
|
|
try {
|
|
runs.push(JSON.parse(line) as Run);
|
|
} catch {
|
|
// A truncated last line — a crash mid-append. Skipped rather than fatal: one unreadable entry
|
|
// must not make the whole history unreadable.
|
|
}
|
|
}
|
|
|
|
return runs.reverse().slice(0, limit);
|
|
}
|