a ledger of runs, and what is still outstanding

Two things the panel could not answer. A run's state lives in a Map, so a
restart erased it — and with it any record that a file FAILED. That is the gap
the manifests structurally cannot cover: a derived folder records the size,
mtime and variants it was built from, so provenance is already precise, but a
file that did not build has no folder and leaves no trace. Once the watcher is
running, a failure at three in the morning would never be discovered.

So one JSON line per run, appended to media/.sync-log.jsonl. A file rather than
a table because db/schema.ts is emphatic that the connection is the only thing
this plugin stores, and a ledger is losable — re-run and it rebuilds. Beside
derived/ rather than inside it, so wiping derived does not erase the history of
what happened before, which is the comparison it exists for. Written in a
finally, so a run that died still records that it did.

And the line above the button now says what is TRUE rather than what a run once
did: reconcile() walks originals, reads each manifest, and reports current,
missing and changed. It uses upToDate — the same predicate the sync uses to skip
a file — because two definitions of "current" is how a panel comes to say
everything is in step while the next run rebuilds half of it. Missing and
changed are separated because only the second means somebody edited something.

The ledger is not polled. It reads the whole tree, so it loads on arrival and
again when a run stops — the only two moments its answer can change.

Sync also takes an optional folder now, so a subtree can be re-synced on its own
rather than the whole library. The scope is recorded on each run, because "3
built" means different things for one folder and for everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-29 11:47:32 +01:00
co-authored by Claude Opus 5
parent 792f166c31
commit f30d40446a
5 changed files with 340 additions and 13 deletions
+74 -2
View File
@@ -151,8 +151,13 @@ async function avClub<T>(path: string, email: string, init?: RequestInit): Promi
const post = <T>(path: string, email: string, body: unknown): Promise<T> =>
avClub<T>(path, email, { method: 'POST', body: JSON.stringify(body) });
/** Is this derived directory already correct for this original? */
async function upToDate(dir: string, source: { bytes: number; mtimeMs: number }): Promise<boolean> {
/**
* Is this derived directory already correct for this original?
*
* Exported because the panel asks the same question the sync does. Two implementations of "current"
* is how a screen comes to say everything is in step while the next run rebuilds half of it.
*/
export async function upToDate(dir: string, source: { bytes: number; mtimeMs: number }): Promise<boolean> {
const raw = await readFile(join(dir, MANIFEST), 'utf8').catch(() => null);
if (!raw) return false;
try {
@@ -410,7 +415,74 @@ export async function listOriginals(root: string, prefix = ''): Promise<string[]
return found;
}
/**
* A caller-supplied subtree of `originals/`, made safe to walk.
*
* Relative, always, and never able to climb out: `..` in any segment is refused rather than resolved,
* because a scope that escapes `originals/` would have the sync building derived folders for files
* nobody put there. Leading and trailing slashes are trimmed so `/GALLERY/` and `GALLERY` mean one
* thing — the caller is naming a folder, not writing a path.
*/
export function safeScope(scope: string): string {
const trimmed = scope.replace(/^\/+|\/+$/g, '');
if (!trimmed) return '';
if (trimmed.split('/').includes('..')) throw new Error('That is not a folder inside originals.');
return trimmed;
}
/** Where a relative original ends up. Exported because the watcher needs the same answer. */
export const targetFor = (derivedRoot: string, relativePath: string): string => join(derivedRoot, relativePath);
export { relative, AvClubUnavailable };
export type Reconciliation = {
/** Convertible files under `originals/`. */
originals: number;
/** Built from the current version of their source. */
current: number;
/** Never built, or built and since lost. Names, so the panel can say WHICH. */
missing: string[];
/** Built, but from a version of the source that has changed since. */
stale: string[];
};
/**
* What `derived/` would still have to do to be in step with `originals/`.
*
* Answered from the filesystem every time rather than cached, because it is the question "is what I am
* looking at current" and a cached answer to that is worse than none.
*
* It uses `upToDate` — the SAME predicate the sync uses to decide whether to skip a file. That is the
* whole point: a panel reporting "62 current" and a run then rebuilding all 62 would be two different
* definitions of the word, and the one on screen would be the one nobody could debug.
*
* `missing` and `stale` are separated because they mean different things to a person. Missing is a file
* that has never been converted or whose output was deleted; stale is one that was replaced with a new
* version. Both are work, but only the second means somebody changed something.
*/
export async function reconcile(email: string, scope = ''): Promise<Reconciliation> {
const from = await originalsDir(email);
const to = await derivedDir(email);
const files = await listOriginals(from, safeScope(scope));
const missing: string[] = [];
const stale: string[] = [];
let current = 0;
for (const file of files) {
const info = await stat(join(from, file)).catch(() => null);
if (!info) continue;
const target = targetFor(to, file);
const manifest = await readFile(join(target, MANIFEST), 'utf8').catch(() => null);
if (!manifest) {
missing.push(file);
continue;
}
if (await upToDate(target, { bytes: info.size, mtimeMs: Math.round(info.mtimeMs) })) current++;
else stale.push(file);
}
return { originals: files.length, current, missing, stale };
}