a ledger of runs, and what is still outstanding

Two things the panel could not answer. A run's state lives in a Map, so a
restart erased it — and with it any record that a file FAILED. That is the gap
the manifests structurally cannot cover: a derived folder records the size,
mtime and variants it was built from, so provenance is already precise, but a
file that did not build has no folder and leaves no trace. Once the watcher is
running, a failure at three in the morning would never be discovered.

So one JSON line per run, appended to media/.sync-log.jsonl. A file rather than
a table because db/schema.ts is emphatic that the connection is the only thing
this plugin stores, and a ledger is losable — re-run and it rebuilds. Beside
derived/ rather than inside it, so wiping derived does not erase the history of
what happened before, which is the comparison it exists for. Written in a
finally, so a run that died still records that it did.

And the line above the button now says what is TRUE rather than what a run once
did: reconcile() walks originals, reads each manifest, and reports current,
missing and changed. It uses upToDate — the same predicate the sync uses to skip
a file — because two definitions of "current" is how a panel comes to say
everything is in step while the next run rebuilds half of it. Missing and
changed are separated because only the second means somebody edited something.

The ledger is not polled. It reads the whole tree, so it loads on arrival and
again when a run stops — the only two moments its answer can change.

Sync also takes an optional folder now, so a subtree can be re-synced on its own
rather than the whole library. The scope is recorded on each run, because "3
built" means different things for one folder and for everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-29 11:47:32 +01:00
co-authored by Claude Opus 5
parent 792f166c31
commit f30d40446a
5 changed files with 340 additions and 13 deletions
+42 -4
View File
@@ -1,6 +1,7 @@
import { mkdir } from 'node:fs/promises';
import { dirname } from 'node:path';
import { deriveOne, derivedDir, listOriginals, originalsDir, targetFor } from './derive';
import { deriveOne, derivedDir, listOriginals, originalsDir, safeScope, targetFor } from './derive';
import { appendRun, type RunTrigger } from './ledger';
/**
* The sync: bring `derived/` into line with `originals/`.
@@ -53,6 +54,14 @@ export type SyncState = {
finishedAt: string | null;
/** Set when the run stopped for a reason that was not about any single file — av-club being down. */
fatal: string | null;
/**
* The subtree this run covered, relative to `originals/`. Empty means all of it.
*
* Recorded rather than implied, because "3 built" means something different depending on whether it
* was the whole library or one folder somebody had just dropped photographs into.
*/
scope: string;
trigger: RunTrigger;
};
/** One. See the note above — this is a memory limit, not a politeness. */
@@ -69,6 +78,8 @@ const idle = (): SyncState => ({
startedAt: null,
finishedAt: null,
fatal: null,
scope: '',
trigger: 'manual',
});
const states = new Map<string, SyncState>();
@@ -81,11 +92,20 @@ export const readSync = (email: string): SyncState => states.get(email) ?? idle(
* Returns the state either way, so the caller does not have to decide what "already running" means to
* a person: the panel shows the run that is in progress, which is the honest answer to "sync now".
*/
export function startSync(email: string): { started: boolean; state: SyncState } {
export function startSync(
email: string,
options: { scope?: string; trigger?: RunTrigger } = {},
): { started: boolean; state: SyncState } {
const existing = states.get(email);
if (existing?.running) return { started: false, state: existing };
const state: SyncState = { ...idle(), running: true, startedAt: new Date().toISOString() };
const state: SyncState = {
...idle(),
running: true,
startedAt: new Date().toISOString(),
scope: safeScope(options.scope ?? ''),
trigger: options.trigger ?? 'manual',
};
states.set(email, state);
// Deliberately not awaited. The route answers immediately with the job's opening state and the panel
@@ -100,7 +120,7 @@ async function run(email: string, state: SyncState): Promise<void> {
const from = await originalsDir(email);
const to = await derivedDir(email);
const files = await listOriginals(from);
const files = await listOriginals(from, state.scope);
state.total = files.length;
let next = 0;
@@ -145,5 +165,23 @@ async function run(email: string, state: SyncState): Promise<void> {
state.running = false;
state.current = [];
state.finishedAt = new Date().toISOString();
/*
The one durable record this leaves.
In `finally`, so a run that died still writes a line — a failure nobody was watching is the whole
reason the ledger exists, and the runs that go unrecorded would be exactly the ones worth having.
*/
await appendRun(email, {
startedAt: state.startedAt ?? state.finishedAt,
finishedAt: state.finishedAt,
trigger: state.trigger,
scope: state.scope,
total: state.total ?? state.done,
built: state.built,
skipped: state.skipped,
errors: state.errors,
fatal: state.fatal,
});
}
}