a ledger of runs, and what is still outstanding
Two things the panel could not answer. A run's state lives in a Map, so a restart erased it — and with it any record that a file FAILED. That is the gap the manifests structurally cannot cover: a derived folder records the size, mtime and variants it was built from, so provenance is already precise, but a file that did not build has no folder and leaves no trace. Once the watcher is running, a failure at three in the morning would never be discovered. So one JSON line per run, appended to media/.sync-log.jsonl. A file rather than a table because db/schema.ts is emphatic that the connection is the only thing this plugin stores, and a ledger is losable — re-run and it rebuilds. Beside derived/ rather than inside it, so wiping derived does not erase the history of what happened before, which is the comparison it exists for. Written in a finally, so a run that died still records that it did. And the line above the button now says what is TRUE rather than what a run once did: reconcile() walks originals, reads each manifest, and reports current, missing and changed. It uses upToDate — the same predicate the sync uses to skip a file — because two definitions of "current" is how a panel comes to say everything is in step while the next run rebuilds half of it. Missing and changed are separated because only the second means somebody edited something. The ledger is not polled. It reads the whole tree, so it loads on arrival and again when a run stops — the only two moments its answer can change. Sync also takes an optional folder now, so a subtree can be re-synced on its own rather than the whole library. The scope is recorded on each run, because "3 built" means different things for one folder and for everything. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
+74
-2
@@ -151,8 +151,13 @@ async function avClub<T>(path: string, email: string, init?: RequestInit): Promi
|
||||
const post = <T>(path: string, email: string, body: unknown): Promise<T> =>
|
||||
avClub<T>(path, email, { method: 'POST', body: JSON.stringify(body) });
|
||||
|
||||
/** Is this derived directory already correct for this original? */
|
||||
async function upToDate(dir: string, source: { bytes: number; mtimeMs: number }): Promise<boolean> {
|
||||
/**
|
||||
* Is this derived directory already correct for this original?
|
||||
*
|
||||
* Exported because the panel asks the same question the sync does. Two implementations of "current"
|
||||
* is how a screen comes to say everything is in step while the next run rebuilds half of it.
|
||||
*/
|
||||
export async function upToDate(dir: string, source: { bytes: number; mtimeMs: number }): Promise<boolean> {
|
||||
const raw = await readFile(join(dir, MANIFEST), 'utf8').catch(() => null);
|
||||
if (!raw) return false;
|
||||
try {
|
||||
@@ -410,7 +415,74 @@ export async function listOriginals(root: string, prefix = ''): Promise<string[]
|
||||
return found;
|
||||
}
|
||||
|
||||
/**
|
||||
* A caller-supplied subtree of `originals/`, made safe to walk.
|
||||
*
|
||||
* Relative, always, and never able to climb out: `..` in any segment is refused rather than resolved,
|
||||
* because a scope that escapes `originals/` would have the sync building derived folders for files
|
||||
* nobody put there. Leading and trailing slashes are trimmed so `/GALLERY/` and `GALLERY` mean one
|
||||
* thing — the caller is naming a folder, not writing a path.
|
||||
*/
|
||||
export function safeScope(scope: string): string {
|
||||
const trimmed = scope.replace(/^\/+|\/+$/g, '');
|
||||
if (!trimmed) return '';
|
||||
if (trimmed.split('/').includes('..')) throw new Error('That is not a folder inside originals.');
|
||||
return trimmed;
|
||||
}
|
||||
|
||||
/** Where a relative original ends up. Exported because the watcher needs the same answer. */
|
||||
export const targetFor = (derivedRoot: string, relativePath: string): string => join(derivedRoot, relativePath);
|
||||
|
||||
export { relative, AvClubUnavailable };
|
||||
|
||||
export type Reconciliation = {
|
||||
/** Convertible files under `originals/`. */
|
||||
originals: number;
|
||||
/** Built from the current version of their source. */
|
||||
current: number;
|
||||
/** Never built, or built and since lost. Names, so the panel can say WHICH. */
|
||||
missing: string[];
|
||||
/** Built, but from a version of the source that has changed since. */
|
||||
stale: string[];
|
||||
};
|
||||
|
||||
/**
|
||||
* What `derived/` would still have to do to be in step with `originals/`.
|
||||
*
|
||||
* Answered from the filesystem every time rather than cached, because it is the question "is what I am
|
||||
* looking at current" and a cached answer to that is worse than none.
|
||||
*
|
||||
* It uses `upToDate` — the SAME predicate the sync uses to decide whether to skip a file. That is the
|
||||
* whole point: a panel reporting "62 current" and a run then rebuilding all 62 would be two different
|
||||
* definitions of the word, and the one on screen would be the one nobody could debug.
|
||||
*
|
||||
* `missing` and `stale` are separated because they mean different things to a person. Missing is a file
|
||||
* that has never been converted or whose output was deleted; stale is one that was replaced with a new
|
||||
* version. Both are work, but only the second means somebody changed something.
|
||||
*/
|
||||
export async function reconcile(email: string, scope = ''): Promise<Reconciliation> {
|
||||
const from = await originalsDir(email);
|
||||
const to = await derivedDir(email);
|
||||
const files = await listOriginals(from, safeScope(scope));
|
||||
|
||||
const missing: string[] = [];
|
||||
const stale: string[] = [];
|
||||
let current = 0;
|
||||
|
||||
for (const file of files) {
|
||||
const info = await stat(join(from, file)).catch(() => null);
|
||||
if (!info) continue;
|
||||
|
||||
const target = targetFor(to, file);
|
||||
const manifest = await readFile(join(target, MANIFEST), 'utf8').catch(() => null);
|
||||
if (!manifest) {
|
||||
missing.push(file);
|
||||
continue;
|
||||
}
|
||||
|
||||
if (await upToDate(target, { bytes: info.size, mtimeMs: Math.round(info.mtimeMs) })) current++;
|
||||
else stale.push(file);
|
||||
}
|
||||
|
||||
return { originals: files.length, current, missing, stale };
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user