Sync says what it is doing, file by file

The counters said how MUCH and never which file, so a run that was working normally and one that had
wedged produced the same screen: `done` stops moving either way. A photograph is around thirty encodes
and a video is three plus a poster ladder, and none of that was visible from outside.

So the run keeps a transcript and the button opens it.

Every rung reports itself by CONSTRUCTION. Ten `variants.push` call sites became one `add`, which is
the point — a rung added later is described without anybody remembering to describe it. The slow
stages that produce nothing until they finish say so BEFORE they start instead: the HEIC decode, each
video rung, the poster frame. Those are the pauses that read as a hang.

`?since=<n>` is what keeps it cheap. The transcript is thousands of lines and polls once a second, so
sending all of it every tick would be the same megabyte over and over; the client keeps what it has
and asks for the rest. No `since` means everything, which is what a modal opened mid-run wants.

The transcript lives in the shared slot, not the modal, so closing it does not throw away the run —
somebody who looks, closes, and looks again is watching one run rather than two. `startedAt` is what
identifies a run, because `n` restarts at 1 and appending across a boundary would interleave two.

Polling, not a socket. Each line transfers once and the events are seconds apart, so a second
transport to open, authenticate and reconnect would buy nothing.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-08-31 12:16:52 +01:00
co-authored by Claude Opus 5
parent 78db209ea1
commit ddcba1c95d
6 changed files with 442 additions and 40 deletions
+54 -13
View File
@@ -538,12 +538,25 @@ async function topUp(dir: string, manifest: Manifest, email: string): Promise<Ma
* finished, because the manifest is what everything else trusts. Half a ladder with a manifest is
* worse than no ladder at all.
*/
/** Told what just happened, rung by rung. See `onStep` — a no-op unless somebody is watching. */
export type Step = (message: string) => void;
export async function deriveOne(
sourcePath: string,
targetDir: string,
email: string,
options: { force?: boolean } = {},
options: { force?: boolean; onStep?: Step } = {},
): Promise<{ built: boolean; manifest: Manifest }> {
/*
Say what is happening, if anybody asked.
A no-op by default, so nothing here depends on being watched — the sync passes one through to fill
its transcript; the watcher and the tests pass nothing. One photograph is around thirty encodes and
a video is three plus a poster ladder, so without this a run that is working normally and a run
that is wedged look identical from outside.
*/
const step = options.onStep ?? (() => {});
const name = sourcePath.slice(sourcePath.lastIndexOf('/') + 1);
const kind = kindOf(name);
if (!kind) throw new Error(`${name} is neither an image nor a video this understands.`);
@@ -563,8 +576,8 @@ export async function deriveOne(
try {
const manifest =
kind === 'image'
? await buildImage(sourcePath, staging, name, source, email)
: await buildVideo(sourcePath, staging, name, source, email);
? await buildImage(sourcePath, staging, name, source, email, step)
: await buildVideo(sourcePath, staging, name, source, email, step);
await writeFile(join(staging, MANIFEST), `${JSON.stringify(manifest, null, 2)}\n`, 'utf8');
@@ -596,6 +609,7 @@ async function buildImage(
name: string,
source: { bytes: number; mtimeMs: number },
email: string,
step: Step,
): Promise<Manifest> {
const meta = await avClub<{ width: number | null; height: number | null; needsConversion: boolean }>(
`/v1/image?path=${encodeURIComponent(sourcePath)}`,
@@ -624,6 +638,9 @@ async function buildImage(
*/
const master = meta.needsConversion ? join(staging, MASTER) : sourcePath;
if (meta.needsConversion) {
// Announced BEFORE it happens, unlike a rung. This is the 8.5-second one, it produces nothing
// anybody sees, and a transcript that went quiet here is the exact case this exists to explain.
step(`decoding ${name} once (${width}x${height})`);
await post('/v1/sharp', email, {
inputPath: sourcePath,
outputPath: master,
@@ -639,7 +656,17 @@ async function buildImage(
*/
const lossless = await prefersLossless(master, staging, email, width);
/*
Collect and announce, so a rung cannot be added without being described.
Ten `variants.push` call sites became one `add`, which is the point: a rung added later reports
itself by construction rather than because somebody remembered to.
*/
const variants: Variant[] = [];
const add = (variant: Variant): void => {
variants.push(variant);
step(`${variant.file} · ${(variant.bytes / 1024).toFixed(0)} KB`);
};
/*
`full.webp` first, and always.
@@ -648,7 +675,7 @@ async function buildImage(
exists at all — the original renders nowhere. A lightbox, a zoom, or handing the client their own
photograph back all need it.
*/
variants.push(await encodeWebp(master, join(staging, 'full.webp'), null, email, width, height, lossless));
add(await encodeWebp(master, join(staging, 'full.webp'), null, email, width, height, lossless));
/*
AVIF beside every rung — except for a picture that wanted the lossless path.
@@ -660,13 +687,13 @@ async function buildImage(
*/
const buildAvif = !lossless;
if (buildAvif) variants.push(await encodeAvif(master, join(staging, 'full.avif'), null, email, width, height));
if (buildAvif) add(await encodeAvif(master, join(staging, 'full.avif'), null, email, width, height));
// Strictly less than the original: a rung at or above it would duplicate `full.webp` byte for byte.
for (const rung of LADDER) {
if (rung >= width) continue;
variants.push(await encodeWebp(master, join(staging, `${rung}.webp`), rung, email, width, height, lossless));
if (buildAvif) variants.push(await encodeAvif(master, join(staging, `${rung}.avif`), rung, email, width, height));
add(await encodeWebp(master, join(staging, `${rung}.webp`), rung, email, width, height, lossless));
if (buildAvif) add(await encodeAvif(master, join(staging, `${rung}.avif`), rung, email, width, height));
}
/*
@@ -686,8 +713,8 @@ async function buildImage(
if (!squareEnough) {
for (const rung of SQUARE_LADDER) {
if (rung > Math.min(width, height)) continue;
variants.push(await encodeSquare(master, join(staging, `${rung}sq.webp`), rung, email, lossless));
if (buildAvif) variants.push(await encodeSquareAvif(master, join(staging, `${rung}sq.avif`), rung, email));
add(await encodeSquare(master, join(staging, `${rung}sq.webp`), rung, email, lossless));
if (buildAvif) add(await encodeSquareAvif(master, join(staging, `${rung}sq.avif`), rung, email));
}
}
@@ -870,6 +897,7 @@ async function buildVideo(
name: string,
source: { bytes: number; mtimeMs: number },
email: string,
step: Step,
): Promise<Manifest> {
const probe = await avClub<{ streams?: Array<{ codec_type?: string; width?: number; height?: number }> }>(
`/v1/probe?path=${encodeURIComponent(sourcePath)}`,
@@ -879,7 +907,17 @@ async function buildVideo(
const height = video?.height ?? 0;
const width = video?.width ?? 0;
/*
Collect and announce, so a rung cannot be added without being described.
Ten `variants.push` call sites became one `add`, which is the point: a rung added later reports
itself by construction rather than because somebody remembered to.
*/
const variants: Variant[] = [];
const add = (variant: Variant): void => {
variants.push(variant);
step(`${variant.file} · ${(variant.bytes / 1024).toFixed(0)} KB`);
};
/*
Every rung the source is actually tall enough for, and the floor when it is tall enough for none.
@@ -892,6 +930,8 @@ async function buildVideo(
const rungs = wanted.length ? wanted : [VIDEO_FLOOR];
for (const rung of rungs) {
// Before, because this is minutes and the line after it is the only other one this loop prints.
step(`encoding ${rung.file}`);
const result = await post<{ path: string; bytes: number }>('/v1/ffmpeg', email, {
inputPath: sourcePath,
outputPath: join(staging, rung.file),
@@ -900,7 +940,7 @@ async function buildVideo(
// see sidecar/ffmpeg/run.ts. Used only to lighten the audio on the 480 rung.
...(rung.options ? { options: rung.options } : {}),
});
variants.push({ file: rung.file, width: null, height: null, bytes: result.bytes });
add({ file: rung.file, width: null, height: null, bytes: result.bytes });
}
/*
@@ -911,6 +951,7 @@ async function buildVideo(
is dropped. Two calls per video, once, at build time.
*/
const stillJpg = join(staging, 'poster.jpg');
step('extracting the poster frame');
await post('/v1/ffmpeg', email, {
inputPath: sourcePath,
outputPath: stillJpg,
@@ -933,7 +974,7 @@ async function buildVideo(
order rather than the other one.
*/
const placeholder = await encodePlaceholder(stillJpg, join(staging, 'placeholder.webp'), email);
variants.push({ file: 'poster.webp', width: width || null, height: height || null, bytes: poster.bytes });
add({ file: 'poster.webp', width: width || null, height: height || null, bytes: poster.bytes });
/*
The poster's own ladder, cut from the same still.
@@ -954,7 +995,7 @@ async function buildVideo(
pipeline: [{ resize: { width: rung, fit: 'inside', withoutEnlargement: true } }],
output: { format: 'webp', options: { quality: WEBP_QUALITY_POSTER } },
});
variants.push({
add({
file: `poster-${rung}.webp`,
width: rung,
height: height ? Math.round((rung / width) * height) : null,
@@ -969,7 +1010,7 @@ async function buildVideo(
pipeline: [{ resize: { width: rung, fit: 'inside', withoutEnlargement: true } }],
output: { format: 'avif', options: { quality: avifQualityFor(rung) } },
});
variants.push({
add({
file: `poster-${rung}.avif`,
width: rung,
height: height ? Math.round((rung / width) * height) : null,
+17 -1
View File
@@ -395,9 +395,25 @@ router.post('/_config/media/sync', async (ctx) => {
}
});
/*
The sync's state, and the part of its transcript the caller does not have yet.
`?since=<n>` is what keeps a modal open on a long run cheap: the transcript is thousands of lines by
the end and polls once a second, so sending all of it every tick would be the same megabyte over and
over. The caller keeps what it has been given and asks for what came after.
Absent `since` means "everything", which is what a modal opening mid-run wants — a watcher that
arrived late should see what already happened rather than start from the next line.
*/
router.get('/_config/media/sync', async (ctx) => {
const user = ctx.get('user');
return ctx.json(readSync(user.email));
const state = readSync(user.email);
const raw = ctx.req.query('since');
const since = raw === undefined ? null : Number(raw);
if (since === null || !Number.isFinite(since)) return ctx.json(state);
return ctx.json({ ...state, log: state.log.filter((line) => line.n > since) });
});
/**
+116 -5
View File
@@ -76,8 +76,45 @@ export type SyncState = {
*/
scope: string;
trigger: RunTrigger;
/**
* A running transcript — what happened, to which file, when.
*
* The counters say how MUCH; this says what. They answer different questions and a run that stalls
* on one photograph is invisible in the counters: `done` simply stops moving, and there is nothing
* to tell "av-club is chewing on a 40-megapixel HEIC" apart from "the process is wedged".
*
* Kept on the state rather than written to a file, because the run already lives in memory and a log
* file would be a second thing to find, rotate and clean up for output whose whole value is that
* somebody is watching it right now. The ledger is what survives; this is what is happening.
*/
log: LogLine[];
};
/** One line of the transcript. */
export type LogLine = {
/**
* Monotonic, and the reason the client can poll cheaply.
*
* A watcher asks for everything after the last number it holds, so a modal open for a long run
* transfers each line once instead of re-sending the whole transcript every second.
*/
n: number;
at: string;
level: 'info' | 'step' | 'done' | 'warn' | 'error';
/** The file this is about, when it is about one. Blank for the run's own lines. */
file: string;
text: string;
};
/**
* How many lines to keep.
*
* A photograph produces about thirty of them, so this is a few hundred files of detail. Past it the
* OLDEST go: the interesting end of a long run is the end, and a transcript that stopped recording at
* the top would drop the failure somebody opened it to read.
*/
const LOG_LIMIT = 4000;
/** One. See the note above — this is a memory limit, not a politeness. */
const CONCURRENCY = 1;
@@ -96,6 +133,7 @@ const idle = (): SyncState => ({
fatal: null,
scope: '',
trigger: 'manual',
log: [],
});
const states = new Map<string, SyncState>();
@@ -131,14 +169,39 @@ export function startSync(
return { started: true, state };
}
/**
* Add a line, and keep the buffer bounded.
*
* `seq` is per-run and starts where the last one left off only in the sense that a new run resets it —
* a watcher that reconnects across runs sees the numbers go backwards, which is why the client drops
* what it holds when `startedAt` changes rather than trusting the numbers alone.
*/
const say = (state: SyncState, level: LogLine['level'], file: string, text: string): void => {
const n = (state.log.at(-1)?.n ?? 0) + 1;
state.log.push({ n, at: new Date().toISOString(), level, file, text });
// Oldest first. See LOG_LIMIT — the end of a long run is the part worth keeping.
if (state.log.length > LOG_LIMIT) state.log.splice(0, state.log.length - LOG_LIMIT);
};
/** `1.4s`, `320ms`. Short enough to sit at the end of a line without becoming the line. */
const since = (from: number): string => {
const ms = Date.now() - from;
return ms < 1000 ? `${ms}ms` : `${(ms / 1000).toFixed(1)}s`;
};
async function run(email: string, state: SyncState): Promise<void> {
const began = Date.now();
try {
const from = await originalsDir(email);
const to = await derivedDir(email);
say(state, 'info', '', `Reading ${state.scope || 'originals/'}`);
const files = await listOriginals(from, state.scope);
state.total = files.length;
say(state, 'info', '', `${files.length} file${files.length === 1 ? '' : 's'} to check`);
/*
What is already built, indexed by what it was built FROM.
@@ -149,6 +212,10 @@ async function run(email: string, state: SyncState): Promise<void> {
*/
const derivedIndex = await indexDerived(to);
// Worth a line: on a big library this is the pause before anything appears to happen, and without
// it the transcript opens with a gap somebody reads as a hang.
say(state, 'info', '', `Indexed ${derivedIndex.size} existing ladder${derivedIndex.size === 1 ? '' : 's'}`);
let next = 0;
const worker = async (): Promise<void> => {
for (;;) {
@@ -157,6 +224,8 @@ async function run(email: string, state: SyncState): Promise<void> {
if (file === undefined) return;
state.current.push(file);
const startedFile = Date.now();
say(state, 'step', file, 'checking');
try {
const target = targetFor(to, file);
await mkdir(dirname(target), { recursive: true });
@@ -175,10 +244,28 @@ async function run(email: string, state: SyncState): Promise<void> {
? await relocateDerived(derivedIndex, to, file, { bytes: info.size, mtimeMs: Math.round(info.mtimeMs) })
: false;
const { built } = await deriveOne(`${from}/${file}`, target, email);
if (relocated && !built) state.moved++;
else if (built) state.built++;
else state.skipped++;
if (relocated) say(state, 'info', file, 'moved — renaming its ladder instead of rebuilding');
/*
Every rung, as it lands. This is the "what is it doing right now" the counters cannot say:
one photograph is thirty encodes and a video is three of them plus a poster ladder, and a
run that looks stopped is usually most of the way through one of those.
*/
const { built } = await deriveOne(`${from}/${file}`, target, email, {
onStep: (message) => say(state, 'step', file, message),
});
if (relocated && !built) {
state.moved++;
say(state, 'done', file, `moved · ${since(startedFile)}`);
} else if (built) {
state.built++;
say(state, 'done', file, `built · ${since(startedFile)}`);
} else {
state.skipped++;
// `current` and not `skipped`: nothing was skipped over, the ladder was already right.
say(state, 'done', file, `already current · ${since(startedFile)}`);
}
} catch (error) {
const message = (error as Error).message;
/*
@@ -188,9 +275,11 @@ async function run(email: string, state: SyncState): Promise<void> {
*/
if (message.includes('av-club is not running')) {
state.fatal = message;
say(state, 'error', '', message);
return;
}
state.errors.push({ file, message });
say(state, 'error', file, message);
} finally {
state.current = state.current.filter((name) => name !== file);
state.done++;
@@ -211,15 +300,37 @@ async function run(email: string, state: SyncState): Promise<void> {
what the name has meant since these two folders stopped being called `user` and `treated`.
*/
if (!state.scope && !state.fatal) {
state.pruned = (await pruneDerived(from, to)).length;
const orphans = await pruneDerived(from, to);
state.pruned = orphans.length;
// Named, not counted. A removal nobody can see the target of is the one thing here somebody
// might want to argue with afterwards.
for (const orphan of orphans) say(state, 'warn', orphan, 'removed — its original is gone');
}
} catch (error) {
state.fatal = (error as Error).message;
say(state, 'error', '', (error as Error).message);
} finally {
state.running = false;
state.current = [];
state.finishedAt = new Date().toISOString();
// The last line, and the one still on screen when the modal stops moving.
say(
state,
state.fatal || state.errors.length ? 'warn' : 'done',
'',
[
`${state.built} built`,
state.skipped ? `${state.skipped} already current` : '',
state.moved ? `${state.moved} moved` : '',
state.pruned ? `${state.pruned} removed` : '',
state.errors.length ? `${state.errors.length} failed` : '',
`in ${since(began)}`,
]
.filter(Boolean)
.join(' · '),
);
/*
The one durable record this leaves.
+33 -18
View File
@@ -1,5 +1,7 @@
import { useState } from 'react';
import { Loader2, RefreshCw } from 'lucide-react';
import { Button } from '@/components/ui/button';
import { SyncModal } from './SyncModal';
import { useMediaSync, useMediaSyncFeed } from './useMediaSync';
import { useReady } from './useReady';
@@ -22,6 +24,13 @@ import { useReady } from './useReady';
* anything in this plugin is open and the Administration card is mounted only sometimes. The thing
* that is always there should be the thing that asks.
*
* ── Pressing it OPENS rather than starts ──
*
* A sync is minutes of work on a folder somebody may not have looked at in a while, and starting it
* from a bar gave no way to see what it was doing — the counters said how much and never which file.
* So the button opens the transcript and the transcript has the verb in it. One press to look, one to
* commit, and the second one is next to the thing it will fill.
*
* ── The label is the state ──
*
* `Sync 3` rather than a badge or a dot: the number IS the reason to press it, and a button that says
@@ -30,8 +39,9 @@ import { useReady } from './useReady';
* is exactly the case somebody needs it for.
*/
export const SyncButton = () => {
const { running, percent, outstanding, busy, start } = useMediaSync();
const { running, percent, outstanding } = useMediaSync();
const { rootSet } = useReady();
const [open, setOpen] = useState(false);
useMediaSyncFeed();
@@ -45,22 +55,27 @@ export const SyncButton = () => {
if (!rootSet) return null;
return (
<Button
size="sm"
variant="outline"
className="h-6 shrink-0 gap-1.5 px-2 text-xs"
onClick={() => void start()}
disabled={busy || running}
title={
running
? 'Building web-ready copies of everything in originals'
: outstanding > 0
? `${outstanding} file${outstanding === 1 ? '' : 's'} in originals have no current copy in derived`
: 'Rebuild anything in derived that is missing or out of date'
}
>
{running ? <Loader2 className="size-3 animate-spin" /> : <RefreshCw className="size-3" />}
{running ? `Syncing${percent === null ? '' : ` ${percent}%`}` : outstanding > 0 ? `Sync ${outstanding}` : 'Sync'}
</Button>
<>
<SyncModal open={open} onClose={() => setOpen(false)} />
<Button
size="sm"
variant="outline"
className="h-6 shrink-0 gap-1.5 px-2 text-xs"
onClick={() => setOpen(true)}
// Never disabled. It opens a window onto a run — which is most worth opening WHILE one is going,
// and a button that locks itself exactly then is the opposite of what it is for.
title={
running
? 'Building web-ready copies of everything in originals'
: outstanding > 0
? `${outstanding} file${outstanding === 1 ? '' : 's'} in originals have no current copy in derived`
: 'Rebuild anything in derived that is missing or out of date'
}
>
{running ? <Loader2 className="size-3 animate-spin" /> : <RefreshCw className="size-3" />}
{running ? `Syncing${percent === null ? '' : ` ${percent}%`}` : outstanding > 0 ? `Sync ${outstanding}` : 'Sync'}
</Button>
</>
);
};
+163
View File
@@ -0,0 +1,163 @@
import { useEffect, useLayoutEffect, useRef, useState } from 'react';
import { Check, Copy, Loader2, RefreshCw } from 'lucide-react';
import { copyToClipboard } from 'helpers/clipboard';
import { Button } from '@/components/ui/button';
import { Dialog, DialogContent, DialogHeader, DialogTitle } from '@/components/ui/dialog';
import { useFlash } from './useFlash';
import { useMediaSync, type LogLine } from './useMediaSync';
/**
* What the sync is doing, line by line, while it does it.
*
* ── Why a transcript and not a progress bar ──
*
* The bar answers "how much" and it was the only answer available. It cannot say WHICH file, and the
* question people actually have during a long run is "is it stuck?" — which looks identical to "it is
* decoding a 40-megapixel HEIC" from outside. A photograph is around thirty encodes and a video is
* three plus a poster ladder; none of that was visible, so a run that was working normally and one
* that had wedged produced the same screen.
*
* ── The lines come from the poll, not a socket ──
*
* The state is already polled once a second while a run is going, and `?since=` means each line is
* transferred once. So this is a stream in the only sense that matters here — lines appear as they
* happen — without a second transport to open, authenticate and reconnect. Sub-second arrival would
* need SSE and would buy nothing: the events themselves are seconds apart.
*
* ── It is not the only place a failure is recorded ──
*
* Closing this loses nothing. The transcript lives in the shared slot and the ledger in Administration
* outlives the run entirely — this is the live view, not the record.
*/
const COLOURS: Record<LogLine['level'], string> = {
info: 'text-muted-foreground',
// The busiest level by far, and the least interesting line at a time. Dimmer than `info` so the
// structure of a run — its starts and its finishes — stays readable as the rungs scroll past.
step: 'text-muted-foreground/60',
done: 'text-foreground',
warn: 'text-amber-500',
error: 'text-destructive',
};
const clock = (iso: string) =>
new Date(iso).toLocaleTimeString(undefined, { hour: '2-digit', minute: '2-digit', second: '2-digit' });
export const SyncModal = ({ open, onClose }: { open: boolean; onClose: () => void }) => {
const { log, state, running, percent, error, start, busy } = useMediaSync();
const [copied, flashCopied] = useFlash(1500);
/*
Follow the tail, unless somebody has scrolled away from it.
A transcript that always jumps to the bottom is unreadable while it is being written: the line
somebody is reading leaves the screen a second after they find it. So this follows only while the
view is already AT the bottom, which is the state it starts in and returns to on its own.
*/
const box = useRef<HTMLDivElement | null>(null);
const [following, setFollowing] = useState(true);
useLayoutEffect(() => {
const node = box.current;
if (!node || !following) return;
node.scrollTop = node.scrollHeight;
}, [log.length, following]);
// Reset to following whenever it opens, so reopening mid-run lands on the newest line rather than
// wherever the last visit left off.
useEffect(() => {
if (open) setFollowing(true);
}, [open]);
const transcript = log.map((line) => `${clock(line.at)} ${line.file ? `${line.file} — ` : ''}${line.text}`).join('\n');
return (
<Dialog open={open} onOpenChange={(next) => !next && onClose()}>
<DialogContent className="flex max-h-[80vh] flex-col gap-3 sm:max-w-3xl">
<DialogHeader>
<DialogTitle className="flex items-center gap-2 text-sm">
{running && <Loader2 className="size-3.5 animate-spin" />}
Sync
<span className="font-normal text-muted-foreground">
{running
? state?.total
? `${state.done} of ${state.total}${percent === null ? '' : ` · ${percent}%`}`
: 'reading originals…'
: state?.finishedAt
? 'finished'
: 'idle'}
</span>
</DialogTitle>
</DialogHeader>
{running && (
<div className="h-1 w-full shrink-0 overflow-hidden rounded-full bg-muted">
{/* Indeterminate until the walk has counted: a bar that starts full and shrinks is worse
than one that has not started. */}
<div
className="h-full rounded-full bg-primary transition-[width] duration-500"
style={{ width: `${percent ?? 0}%` }}
/>
</div>
)}
<div
ref={box}
// `onScroll` rather than a button: following is a thing somebody stops by scrolling, which is
// what they were already doing, and resumes by scrolling back down.
onScroll={(event) => {
const node = event.currentTarget;
setFollowing(node.scrollHeight - node.scrollTop - node.clientHeight < 24);
}}
className="min-h-[16rem] flex-1 overflow-auto rounded-md border border-border bg-muted/30 p-3 font-mono text-[11px] leading-relaxed"
>
{log.length === 0 ? (
<p className="text-muted-foreground">
Nothing yet. Press Sync to build web-ready copies of everything in originals.
</p>
) : (
log.map((line) => (
<div key={line.n} className={`flex gap-2 ${COLOURS[line.level]}`}>
<span className="shrink-0 opacity-50">{clock(line.at)}</span>
{/* The file is not repeated on every rung of the same file — the eye follows a column
of identical names badly, and the interesting part is what changed. */}
{line.file && <span className="min-w-0 shrink-0 max-w-[45%] truncate">{line.file}</span>}
<span className="min-w-0 flex-1 break-words">{line.text}</span>
</div>
))
)}
</div>
{error && <p className="shrink-0 text-xs text-destructive">{error}</p>}
<div className="flex shrink-0 items-center justify-between gap-2">
<Button
size="sm"
variant="outline"
className="h-7 gap-1.5 text-xs"
disabled={log.length === 0}
onClick={() => {
void copyToClipboard(transcript);
flashCopied();
}}
>
{copied ? <Check className="size-3" /> : <Copy className="size-3" />}
{copied ? 'Copied' : 'Copy'}
</Button>
{/* Start from in here too. Opening the modal to watch and then having to close it to press
the button is the sort of thing that only reads as obvious to whoever built it. */}
<Button
size="sm"
className="h-7 gap-1.5 text-xs"
onClick={() => void start()}
disabled={busy || running}
>
{running ? <Loader2 className="size-3 animate-spin" /> : <RefreshCw className="size-3" />}
{running ? 'Syncing' : 'Sync'}
</Button>
</div>
</DialogContent>
</Dialog>
);
};
+59 -3
View File
@@ -31,6 +31,17 @@ import { readableError } from './errors';
export type SyncError = { file: string; message: string };
/** One line of the run's transcript. Mirrors LogLine in api/sync.ts. */
export type LogLine = {
/** Monotonic within a run. What `?since=` is asked with, so each line transfers once. */
n: number;
at: string;
level: 'info' | 'step' | 'done' | 'warn' | 'error';
/** The file this is about, or blank for the run's own lines. */
file: string;
text: string;
};
export type SyncState = {
running: boolean;
total: number | null;
@@ -43,6 +54,8 @@ export type SyncState = {
pruned: number;
errors: SyncError[];
current: string[];
/** Only the lines the server had not already sent — see the `since` note in the feed. */
log: LogLine[];
startedAt: string | null;
finishedAt: string | null;
fatal: string | null;
@@ -75,12 +88,21 @@ const KEY = ['fofos-admin', 'media-sync'];
type Shared = {
state: SyncState | null;
ledger: Ledger | null;
/**
* The transcript, ACCUMULATED — the server sends deltas and this is the whole of it.
*
* Kept here rather than in the modal so closing the modal does not throw away the run's history.
* Somebody who opens it, watches, closes it and opens it again is watching one run, not two.
*/
log: LogLine[];
/** Which run `log` belongs to, so a new one clears it. See the note where it is compared. */
logRun: string | null;
/** A start request is in flight — distinct from `state.running`, which is the SERVER's answer. */
busy: boolean;
error: string | null;
};
const EMPTY: Shared = { state: null, ledger: null, busy: false, error: null };
const EMPTY: Shared = { state: null, ledger: null, log: [], logRun: null, busy: false, error: null };
export type MediaSync = Shared & {
/** A pass is under way. */
@@ -147,10 +169,44 @@ export const useMediaSyncFeed = (): void => {
const setRef = useRef(setShared);
setRef.current = setShared;
// The latest value, for the poll to read WITHOUT being rebuilt when it changes — an interval whose
// callback identity changed every line would be torn down and restarted on every line.
const sharedRef = useRef(shared);
sharedRef.current = shared;
const load = useCallback(async () => {
const held = sharedRef.current;
const since = held.log.at(-1)?.n;
try {
const state = await clientRef.current.get<SyncState>('/fofos-admin/_config/media/sync');
setRef.current((was) => ({ ...was, state }));
/*
Ask only for what we do not have.
A long run's transcript is thousands of lines and this polls every second, so without `since`
the same megabyte would arrive over and over. On the first poll of a run there is nothing held,
so `since` is absent and the server sends the lot — which is what a modal opened mid-run wants.
*/
const state = await clientRef.current.get<SyncState>(
since === undefined
? '/fofos-admin/_config/media/sync'
: `/fofos-admin/_config/media/sync?since=${since}`,
);
setRef.current((was) => {
/*
A NEW run resets the transcript.
`n` restarts at 1 each run, so appending across a boundary would interleave two runs and put
the numbers out of order. `startedAt` is what identifies a run — comparing it is cheaper and
more honest than watching `running` go false then true, which misses a run that began and
ended between two polls.
*/
const run = state.startedAt;
const fresh = run !== was.logRun;
const log = fresh ? state.log : [...was.log, ...state.log];
return { ...was, state, log, logRun: run };
});
} catch {
// Owner-only. Root Location has already said so once if that is the reason.
setRef.current((was) => ({ ...was, state: null }));