Sync says what it is doing, file by file
The counters said how MUCH and never which file, so a run that was working normally and one that had wedged produced the same screen: `done` stops moving either way. A photograph is around thirty encodes and a video is three plus a poster ladder, and none of that was visible from outside. So the run keeps a transcript and the button opens it. Every rung reports itself by CONSTRUCTION. Ten `variants.push` call sites became one `add`, which is the point — a rung added later is described without anybody remembering to describe it. The slow stages that produce nothing until they finish say so BEFORE they start instead: the HEIC decode, each video rung, the poster frame. Those are the pauses that read as a hang. `?since=<n>` is what keeps it cheap. The transcript is thousands of lines and polls once a second, so sending all of it every tick would be the same megabyte over and over; the client keeps what it has and asks for the rest. No `since` means everything, which is what a modal opened mid-run wants. The transcript lives in the shared slot, not the modal, so closing it does not throw away the run — somebody who looks, closes, and looks again is watching one run rather than two. `startedAt` is what identifies a run, because `n` restarts at 1 and appending across a boundary would interleave two. Polling, not a socket. Each line transfers once and the events are seconds apart, so a second transport to open, authenticate and reconnect would buy nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+54
-13
@@ -538,12 +538,25 @@ async function topUp(dir: string, manifest: Manifest, email: string): Promise<Ma
|
||||
* finished, because the manifest is what everything else trusts. Half a ladder with a manifest is
|
||||
* worse than no ladder at all.
|
||||
*/
|
||||
/** Told what just happened, rung by rung. See `onStep` — a no-op unless somebody is watching. */
|
||||
export type Step = (message: string) => void;
|
||||
|
||||
export async function deriveOne(
|
||||
sourcePath: string,
|
||||
targetDir: string,
|
||||
email: string,
|
||||
options: { force?: boolean } = {},
|
||||
options: { force?: boolean; onStep?: Step } = {},
|
||||
): Promise<{ built: boolean; manifest: Manifest }> {
|
||||
/*
|
||||
Say what is happening, if anybody asked.
|
||||
|
||||
A no-op by default, so nothing here depends on being watched — the sync passes one through to fill
|
||||
its transcript; the watcher and the tests pass nothing. One photograph is around thirty encodes and
|
||||
a video is three plus a poster ladder, so without this a run that is working normally and a run
|
||||
that is wedged look identical from outside.
|
||||
*/
|
||||
const step = options.onStep ?? (() => {});
|
||||
|
||||
const name = sourcePath.slice(sourcePath.lastIndexOf('/') + 1);
|
||||
const kind = kindOf(name);
|
||||
if (!kind) throw new Error(`${name} is neither an image nor a video this understands.`);
|
||||
@@ -563,8 +576,8 @@ export async function deriveOne(
|
||||
try {
|
||||
const manifest =
|
||||
kind === 'image'
|
||||
? await buildImage(sourcePath, staging, name, source, email)
|
||||
: await buildVideo(sourcePath, staging, name, source, email);
|
||||
? await buildImage(sourcePath, staging, name, source, email, step)
|
||||
: await buildVideo(sourcePath, staging, name, source, email, step);
|
||||
|
||||
await writeFile(join(staging, MANIFEST), `${JSON.stringify(manifest, null, 2)}\n`, 'utf8');
|
||||
|
||||
@@ -596,6 +609,7 @@ async function buildImage(
|
||||
name: string,
|
||||
source: { bytes: number; mtimeMs: number },
|
||||
email: string,
|
||||
step: Step,
|
||||
): Promise<Manifest> {
|
||||
const meta = await avClub<{ width: number | null; height: number | null; needsConversion: boolean }>(
|
||||
`/v1/image?path=${encodeURIComponent(sourcePath)}`,
|
||||
@@ -624,6 +638,9 @@ async function buildImage(
|
||||
*/
|
||||
const master = meta.needsConversion ? join(staging, MASTER) : sourcePath;
|
||||
if (meta.needsConversion) {
|
||||
// Announced BEFORE it happens, unlike a rung. This is the 8.5-second one, it produces nothing
|
||||
// anybody sees, and a transcript that went quiet here is the exact case this exists to explain.
|
||||
step(`decoding ${name} once (${width}x${height})`);
|
||||
await post('/v1/sharp', email, {
|
||||
inputPath: sourcePath,
|
||||
outputPath: master,
|
||||
@@ -639,7 +656,17 @@ async function buildImage(
|
||||
*/
|
||||
const lossless = await prefersLossless(master, staging, email, width);
|
||||
|
||||
/*
|
||||
Collect and announce, so a rung cannot be added without being described.
|
||||
|
||||
Ten `variants.push` call sites became one `add`, which is the point: a rung added later reports
|
||||
itself by construction rather than because somebody remembered to.
|
||||
*/
|
||||
const variants: Variant[] = [];
|
||||
const add = (variant: Variant): void => {
|
||||
variants.push(variant);
|
||||
step(`${variant.file} · ${(variant.bytes / 1024).toFixed(0)} KB`);
|
||||
};
|
||||
|
||||
/*
|
||||
`full.webp` first, and always.
|
||||
@@ -648,7 +675,7 @@ async function buildImage(
|
||||
exists at all — the original renders nowhere. A lightbox, a zoom, or handing the client their own
|
||||
photograph back all need it.
|
||||
*/
|
||||
variants.push(await encodeWebp(master, join(staging, 'full.webp'), null, email, width, height, lossless));
|
||||
add(await encodeWebp(master, join(staging, 'full.webp'), null, email, width, height, lossless));
|
||||
|
||||
/*
|
||||
AVIF beside every rung — except for a picture that wanted the lossless path.
|
||||
@@ -660,13 +687,13 @@ async function buildImage(
|
||||
*/
|
||||
const buildAvif = !lossless;
|
||||
|
||||
if (buildAvif) variants.push(await encodeAvif(master, join(staging, 'full.avif'), null, email, width, height));
|
||||
if (buildAvif) add(await encodeAvif(master, join(staging, 'full.avif'), null, email, width, height));
|
||||
|
||||
// Strictly less than the original: a rung at or above it would duplicate `full.webp` byte for byte.
|
||||
for (const rung of LADDER) {
|
||||
if (rung >= width) continue;
|
||||
variants.push(await encodeWebp(master, join(staging, `${rung}.webp`), rung, email, width, height, lossless));
|
||||
if (buildAvif) variants.push(await encodeAvif(master, join(staging, `${rung}.avif`), rung, email, width, height));
|
||||
add(await encodeWebp(master, join(staging, `${rung}.webp`), rung, email, width, height, lossless));
|
||||
if (buildAvif) add(await encodeAvif(master, join(staging, `${rung}.avif`), rung, email, width, height));
|
||||
}
|
||||
|
||||
/*
|
||||
@@ -686,8 +713,8 @@ async function buildImage(
|
||||
if (!squareEnough) {
|
||||
for (const rung of SQUARE_LADDER) {
|
||||
if (rung > Math.min(width, height)) continue;
|
||||
variants.push(await encodeSquare(master, join(staging, `${rung}sq.webp`), rung, email, lossless));
|
||||
if (buildAvif) variants.push(await encodeSquareAvif(master, join(staging, `${rung}sq.avif`), rung, email));
|
||||
add(await encodeSquare(master, join(staging, `${rung}sq.webp`), rung, email, lossless));
|
||||
if (buildAvif) add(await encodeSquareAvif(master, join(staging, `${rung}sq.avif`), rung, email));
|
||||
}
|
||||
}
|
||||
|
||||
@@ -870,6 +897,7 @@ async function buildVideo(
|
||||
name: string,
|
||||
source: { bytes: number; mtimeMs: number },
|
||||
email: string,
|
||||
step: Step,
|
||||
): Promise<Manifest> {
|
||||
const probe = await avClub<{ streams?: Array<{ codec_type?: string; width?: number; height?: number }> }>(
|
||||
`/v1/probe?path=${encodeURIComponent(sourcePath)}`,
|
||||
@@ -879,7 +907,17 @@ async function buildVideo(
|
||||
const height = video?.height ?? 0;
|
||||
const width = video?.width ?? 0;
|
||||
|
||||
/*
|
||||
Collect and announce, so a rung cannot be added without being described.
|
||||
|
||||
Ten `variants.push` call sites became one `add`, which is the point: a rung added later reports
|
||||
itself by construction rather than because somebody remembered to.
|
||||
*/
|
||||
const variants: Variant[] = [];
|
||||
const add = (variant: Variant): void => {
|
||||
variants.push(variant);
|
||||
step(`${variant.file} · ${(variant.bytes / 1024).toFixed(0)} KB`);
|
||||
};
|
||||
|
||||
/*
|
||||
Every rung the source is actually tall enough for, and the floor when it is tall enough for none.
|
||||
@@ -892,6 +930,8 @@ async function buildVideo(
|
||||
const rungs = wanted.length ? wanted : [VIDEO_FLOOR];
|
||||
|
||||
for (const rung of rungs) {
|
||||
// Before, because this is minutes and the line after it is the only other one this loop prints.
|
||||
step(`encoding ${rung.file}`);
|
||||
const result = await post<{ path: string; bytes: number }>('/v1/ffmpeg', email, {
|
||||
inputPath: sourcePath,
|
||||
outputPath: join(staging, rung.file),
|
||||
@@ -900,7 +940,7 @@ async function buildVideo(
|
||||
// see sidecar/ffmpeg/run.ts. Used only to lighten the audio on the 480 rung.
|
||||
...(rung.options ? { options: rung.options } : {}),
|
||||
});
|
||||
variants.push({ file: rung.file, width: null, height: null, bytes: result.bytes });
|
||||
add({ file: rung.file, width: null, height: null, bytes: result.bytes });
|
||||
}
|
||||
|
||||
/*
|
||||
@@ -911,6 +951,7 @@ async function buildVideo(
|
||||
is dropped. Two calls per video, once, at build time.
|
||||
*/
|
||||
const stillJpg = join(staging, 'poster.jpg');
|
||||
step('extracting the poster frame');
|
||||
await post('/v1/ffmpeg', email, {
|
||||
inputPath: sourcePath,
|
||||
outputPath: stillJpg,
|
||||
@@ -933,7 +974,7 @@ async function buildVideo(
|
||||
order rather than the other one.
|
||||
*/
|
||||
const placeholder = await encodePlaceholder(stillJpg, join(staging, 'placeholder.webp'), email);
|
||||
variants.push({ file: 'poster.webp', width: width || null, height: height || null, bytes: poster.bytes });
|
||||
add({ file: 'poster.webp', width: width || null, height: height || null, bytes: poster.bytes });
|
||||
|
||||
/*
|
||||
The poster's own ladder, cut from the same still.
|
||||
@@ -954,7 +995,7 @@ async function buildVideo(
|
||||
pipeline: [{ resize: { width: rung, fit: 'inside', withoutEnlargement: true } }],
|
||||
output: { format: 'webp', options: { quality: WEBP_QUALITY_POSTER } },
|
||||
});
|
||||
variants.push({
|
||||
add({
|
||||
file: `poster-${rung}.webp`,
|
||||
width: rung,
|
||||
height: height ? Math.round((rung / width) * height) : null,
|
||||
@@ -969,7 +1010,7 @@ async function buildVideo(
|
||||
pipeline: [{ resize: { width: rung, fit: 'inside', withoutEnlargement: true } }],
|
||||
output: { format: 'avif', options: { quality: avifQualityFor(rung) } },
|
||||
});
|
||||
variants.push({
|
||||
add({
|
||||
file: `poster-${rung}.avif`,
|
||||
width: rung,
|
||||
height: height ? Math.round((rung / width) * height) : null,
|
||||
|
||||
+17
-1
@@ -395,9 +395,25 @@ router.post('/_config/media/sync', async (ctx) => {
|
||||
}
|
||||
});
|
||||
|
||||
/*
|
||||
The sync's state, and the part of its transcript the caller does not have yet.
|
||||
|
||||
`?since=<n>` is what keeps a modal open on a long run cheap: the transcript is thousands of lines by
|
||||
the end and polls once a second, so sending all of it every tick would be the same megabyte over and
|
||||
over. The caller keeps what it has been given and asks for what came after.
|
||||
|
||||
Absent `since` means "everything", which is what a modal opening mid-run wants — a watcher that
|
||||
arrived late should see what already happened rather than start from the next line.
|
||||
*/
|
||||
router.get('/_config/media/sync', async (ctx) => {
|
||||
const user = ctx.get('user');
|
||||
return ctx.json(readSync(user.email));
|
||||
const state = readSync(user.email);
|
||||
|
||||
const raw = ctx.req.query('since');
|
||||
const since = raw === undefined ? null : Number(raw);
|
||||
if (since === null || !Number.isFinite(since)) return ctx.json(state);
|
||||
|
||||
return ctx.json({ ...state, log: state.log.filter((line) => line.n > since) });
|
||||
});
|
||||
|
||||
/**
|
||||
|
||||
+116
-5
@@ -76,8 +76,45 @@ export type SyncState = {
|
||||
*/
|
||||
scope: string;
|
||||
trigger: RunTrigger;
|
||||
/**
|
||||
* A running transcript — what happened, to which file, when.
|
||||
*
|
||||
* The counters say how MUCH; this says what. They answer different questions and a run that stalls
|
||||
* on one photograph is invisible in the counters: `done` simply stops moving, and there is nothing
|
||||
* to tell "av-club is chewing on a 40-megapixel HEIC" apart from "the process is wedged".
|
||||
*
|
||||
* Kept on the state rather than written to a file, because the run already lives in memory and a log
|
||||
* file would be a second thing to find, rotate and clean up for output whose whole value is that
|
||||
* somebody is watching it right now. The ledger is what survives; this is what is happening.
|
||||
*/
|
||||
log: LogLine[];
|
||||
};
|
||||
|
||||
/** One line of the transcript. */
|
||||
export type LogLine = {
|
||||
/**
|
||||
* Monotonic, and the reason the client can poll cheaply.
|
||||
*
|
||||
* A watcher asks for everything after the last number it holds, so a modal open for a long run
|
||||
* transfers each line once instead of re-sending the whole transcript every second.
|
||||
*/
|
||||
n: number;
|
||||
at: string;
|
||||
level: 'info' | 'step' | 'done' | 'warn' | 'error';
|
||||
/** The file this is about, when it is about one. Blank for the run's own lines. */
|
||||
file: string;
|
||||
text: string;
|
||||
};
|
||||
|
||||
/**
|
||||
* How many lines to keep.
|
||||
*
|
||||
* A photograph produces about thirty of them, so this is a few hundred files of detail. Past it the
|
||||
* OLDEST go: the interesting end of a long run is the end, and a transcript that stopped recording at
|
||||
* the top would drop the failure somebody opened it to read.
|
||||
*/
|
||||
const LOG_LIMIT = 4000;
|
||||
|
||||
/** One. See the note above — this is a memory limit, not a politeness. */
|
||||
const CONCURRENCY = 1;
|
||||
|
||||
@@ -96,6 +133,7 @@ const idle = (): SyncState => ({
|
||||
fatal: null,
|
||||
scope: '',
|
||||
trigger: 'manual',
|
||||
log: [],
|
||||
});
|
||||
|
||||
const states = new Map<string, SyncState>();
|
||||
@@ -131,14 +169,39 @@ export function startSync(
|
||||
return { started: true, state };
|
||||
}
|
||||
|
||||
/**
|
||||
* Add a line, and keep the buffer bounded.
|
||||
*
|
||||
* `seq` is per-run and starts where the last one left off only in the sense that a new run resets it —
|
||||
* a watcher that reconnects across runs sees the numbers go backwards, which is why the client drops
|
||||
* what it holds when `startedAt` changes rather than trusting the numbers alone.
|
||||
*/
|
||||
const say = (state: SyncState, level: LogLine['level'], file: string, text: string): void => {
|
||||
const n = (state.log.at(-1)?.n ?? 0) + 1;
|
||||
state.log.push({ n, at: new Date().toISOString(), level, file, text });
|
||||
// Oldest first. See LOG_LIMIT — the end of a long run is the part worth keeping.
|
||||
if (state.log.length > LOG_LIMIT) state.log.splice(0, state.log.length - LOG_LIMIT);
|
||||
};
|
||||
|
||||
/** `1.4s`, `320ms`. Short enough to sit at the end of a line without becoming the line. */
|
||||
const since = (from: number): string => {
|
||||
const ms = Date.now() - from;
|
||||
return ms < 1000 ? `${ms}ms` : `${(ms / 1000).toFixed(1)}s`;
|
||||
};
|
||||
|
||||
async function run(email: string, state: SyncState): Promise<void> {
|
||||
const began = Date.now();
|
||||
try {
|
||||
const from = await originalsDir(email);
|
||||
const to = await derivedDir(email);
|
||||
|
||||
say(state, 'info', '', `Reading ${state.scope || 'originals/'}`);
|
||||
|
||||
const files = await listOriginals(from, state.scope);
|
||||
state.total = files.length;
|
||||
|
||||
say(state, 'info', '', `${files.length} file${files.length === 1 ? '' : 's'} to check`);
|
||||
|
||||
/*
|
||||
What is already built, indexed by what it was built FROM.
|
||||
|
||||
@@ -149,6 +212,10 @@ async function run(email: string, state: SyncState): Promise<void> {
|
||||
*/
|
||||
const derivedIndex = await indexDerived(to);
|
||||
|
||||
// Worth a line: on a big library this is the pause before anything appears to happen, and without
|
||||
// it the transcript opens with a gap somebody reads as a hang.
|
||||
say(state, 'info', '', `Indexed ${derivedIndex.size} existing ladder${derivedIndex.size === 1 ? '' : 's'}`);
|
||||
|
||||
let next = 0;
|
||||
const worker = async (): Promise<void> => {
|
||||
for (;;) {
|
||||
@@ -157,6 +224,8 @@ async function run(email: string, state: SyncState): Promise<void> {
|
||||
if (file === undefined) return;
|
||||
|
||||
state.current.push(file);
|
||||
const startedFile = Date.now();
|
||||
say(state, 'step', file, 'checking');
|
||||
try {
|
||||
const target = targetFor(to, file);
|
||||
await mkdir(dirname(target), { recursive: true });
|
||||
@@ -175,10 +244,28 @@ async function run(email: string, state: SyncState): Promise<void> {
|
||||
? await relocateDerived(derivedIndex, to, file, { bytes: info.size, mtimeMs: Math.round(info.mtimeMs) })
|
||||
: false;
|
||||
|
||||
const { built } = await deriveOne(`${from}/${file}`, target, email);
|
||||
if (relocated && !built) state.moved++;
|
||||
else if (built) state.built++;
|
||||
else state.skipped++;
|
||||
if (relocated) say(state, 'info', file, 'moved — renaming its ladder instead of rebuilding');
|
||||
|
||||
/*
|
||||
Every rung, as it lands. This is the "what is it doing right now" the counters cannot say:
|
||||
one photograph is thirty encodes and a video is three of them plus a poster ladder, and a
|
||||
run that looks stopped is usually most of the way through one of those.
|
||||
*/
|
||||
const { built } = await deriveOne(`${from}/${file}`, target, email, {
|
||||
onStep: (message) => say(state, 'step', file, message),
|
||||
});
|
||||
|
||||
if (relocated && !built) {
|
||||
state.moved++;
|
||||
say(state, 'done', file, `moved · ${since(startedFile)}`);
|
||||
} else if (built) {
|
||||
state.built++;
|
||||
say(state, 'done', file, `built · ${since(startedFile)}`);
|
||||
} else {
|
||||
state.skipped++;
|
||||
// `current` and not `skipped`: nothing was skipped over, the ladder was already right.
|
||||
say(state, 'done', file, `already current · ${since(startedFile)}`);
|
||||
}
|
||||
} catch (error) {
|
||||
const message = (error as Error).message;
|
||||
/*
|
||||
@@ -188,9 +275,11 @@ async function run(email: string, state: SyncState): Promise<void> {
|
||||
*/
|
||||
if (message.includes('av-club is not running')) {
|
||||
state.fatal = message;
|
||||
say(state, 'error', '', message);
|
||||
return;
|
||||
}
|
||||
state.errors.push({ file, message });
|
||||
say(state, 'error', file, message);
|
||||
} finally {
|
||||
state.current = state.current.filter((name) => name !== file);
|
||||
state.done++;
|
||||
@@ -211,15 +300,37 @@ async function run(email: string, state: SyncState): Promise<void> {
|
||||
what the name has meant since these two folders stopped being called `user` and `treated`.
|
||||
*/
|
||||
if (!state.scope && !state.fatal) {
|
||||
state.pruned = (await pruneDerived(from, to)).length;
|
||||
const orphans = await pruneDerived(from, to);
|
||||
state.pruned = orphans.length;
|
||||
// Named, not counted. A removal nobody can see the target of is the one thing here somebody
|
||||
// might want to argue with afterwards.
|
||||
for (const orphan of orphans) say(state, 'warn', orphan, 'removed — its original is gone');
|
||||
}
|
||||
} catch (error) {
|
||||
state.fatal = (error as Error).message;
|
||||
say(state, 'error', '', (error as Error).message);
|
||||
} finally {
|
||||
state.running = false;
|
||||
state.current = [];
|
||||
state.finishedAt = new Date().toISOString();
|
||||
|
||||
// The last line, and the one still on screen when the modal stops moving.
|
||||
say(
|
||||
state,
|
||||
state.fatal || state.errors.length ? 'warn' : 'done',
|
||||
'',
|
||||
[
|
||||
`${state.built} built`,
|
||||
state.skipped ? `${state.skipped} already current` : '',
|
||||
state.moved ? `${state.moved} moved` : '',
|
||||
state.pruned ? `${state.pruned} removed` : '',
|
||||
state.errors.length ? `${state.errors.length} failed` : '',
|
||||
`in ${since(began)}`,
|
||||
]
|
||||
.filter(Boolean)
|
||||
.join(' · '),
|
||||
);
|
||||
|
||||
/*
|
||||
The one durable record this leaves.
|
||||
|
||||
|
||||
+18
-3
@@ -1,5 +1,7 @@
|
||||
import { useState } from 'react';
|
||||
import { Loader2, RefreshCw } from 'lucide-react';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { SyncModal } from './SyncModal';
|
||||
import { useMediaSync, useMediaSyncFeed } from './useMediaSync';
|
||||
import { useReady } from './useReady';
|
||||
|
||||
@@ -22,6 +24,13 @@ import { useReady } from './useReady';
|
||||
* anything in this plugin is open and the Administration card is mounted only sometimes. The thing
|
||||
* that is always there should be the thing that asks.
|
||||
*
|
||||
* ── Pressing it OPENS rather than starts ──
|
||||
*
|
||||
* A sync is minutes of work on a folder somebody may not have looked at in a while, and starting it
|
||||
* from a bar gave no way to see what it was doing — the counters said how much and never which file.
|
||||
* So the button opens the transcript and the transcript has the verb in it. One press to look, one to
|
||||
* commit, and the second one is next to the thing it will fill.
|
||||
*
|
||||
* ── The label is the state ──
|
||||
*
|
||||
* `Sync 3` rather than a badge or a dot: the number IS the reason to press it, and a button that says
|
||||
@@ -30,8 +39,9 @@ import { useReady } from './useReady';
|
||||
* is exactly the case somebody needs it for.
|
||||
*/
|
||||
export const SyncButton = () => {
|
||||
const { running, percent, outstanding, busy, start } = useMediaSync();
|
||||
const { running, percent, outstanding } = useMediaSync();
|
||||
const { rootSet } = useReady();
|
||||
const [open, setOpen] = useState(false);
|
||||
|
||||
useMediaSyncFeed();
|
||||
|
||||
@@ -45,12 +55,16 @@ export const SyncButton = () => {
|
||||
if (!rootSet) return null;
|
||||
|
||||
return (
|
||||
<>
|
||||
<SyncModal open={open} onClose={() => setOpen(false)} />
|
||||
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
className="h-6 shrink-0 gap-1.5 px-2 text-xs"
|
||||
onClick={() => void start()}
|
||||
disabled={busy || running}
|
||||
onClick={() => setOpen(true)}
|
||||
// Never disabled. It opens a window onto a run — which is most worth opening WHILE one is going,
|
||||
// and a button that locks itself exactly then is the opposite of what it is for.
|
||||
title={
|
||||
running
|
||||
? 'Building web-ready copies of everything in originals'
|
||||
@@ -62,5 +76,6 @@ export const SyncButton = () => {
|
||||
{running ? <Loader2 className="size-3 animate-spin" /> : <RefreshCw className="size-3" />}
|
||||
{running ? `Syncing${percent === null ? '' : ` ${percent}%`}` : outstanding > 0 ? `Sync ${outstanding}` : 'Sync'}
|
||||
</Button>
|
||||
</>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -0,0 +1,163 @@
|
||||
import { useEffect, useLayoutEffect, useRef, useState } from 'react';
|
||||
import { Check, Copy, Loader2, RefreshCw } from 'lucide-react';
|
||||
import { copyToClipboard } from 'helpers/clipboard';
|
||||
import { Button } from '@/components/ui/button';
|
||||
import { Dialog, DialogContent, DialogHeader, DialogTitle } from '@/components/ui/dialog';
|
||||
import { useFlash } from './useFlash';
|
||||
import { useMediaSync, type LogLine } from './useMediaSync';
|
||||
|
||||
/**
|
||||
* What the sync is doing, line by line, while it does it.
|
||||
*
|
||||
* ── Why a transcript and not a progress bar ──
|
||||
*
|
||||
* The bar answers "how much" and it was the only answer available. It cannot say WHICH file, and the
|
||||
* question people actually have during a long run is "is it stuck?" — which looks identical to "it is
|
||||
* decoding a 40-megapixel HEIC" from outside. A photograph is around thirty encodes and a video is
|
||||
* three plus a poster ladder; none of that was visible, so a run that was working normally and one
|
||||
* that had wedged produced the same screen.
|
||||
*
|
||||
* ── The lines come from the poll, not a socket ──
|
||||
*
|
||||
* The state is already polled once a second while a run is going, and `?since=` means each line is
|
||||
* transferred once. So this is a stream in the only sense that matters here — lines appear as they
|
||||
* happen — without a second transport to open, authenticate and reconnect. Sub-second arrival would
|
||||
* need SSE and would buy nothing: the events themselves are seconds apart.
|
||||
*
|
||||
* ── It is not the only place a failure is recorded ──
|
||||
*
|
||||
* Closing this loses nothing. The transcript lives in the shared slot and the ledger in Administration
|
||||
* outlives the run entirely — this is the live view, not the record.
|
||||
*/
|
||||
|
||||
const COLOURS: Record<LogLine['level'], string> = {
|
||||
info: 'text-muted-foreground',
|
||||
// The busiest level by far, and the least interesting line at a time. Dimmer than `info` so the
|
||||
// structure of a run — its starts and its finishes — stays readable as the rungs scroll past.
|
||||
step: 'text-muted-foreground/60',
|
||||
done: 'text-foreground',
|
||||
warn: 'text-amber-500',
|
||||
error: 'text-destructive',
|
||||
};
|
||||
|
||||
const clock = (iso: string) =>
|
||||
new Date(iso).toLocaleTimeString(undefined, { hour: '2-digit', minute: '2-digit', second: '2-digit' });
|
||||
|
||||
export const SyncModal = ({ open, onClose }: { open: boolean; onClose: () => void }) => {
|
||||
const { log, state, running, percent, error, start, busy } = useMediaSync();
|
||||
const [copied, flashCopied] = useFlash(1500);
|
||||
|
||||
/*
|
||||
Follow the tail, unless somebody has scrolled away from it.
|
||||
|
||||
A transcript that always jumps to the bottom is unreadable while it is being written: the line
|
||||
somebody is reading leaves the screen a second after they find it. So this follows only while the
|
||||
view is already AT the bottom, which is the state it starts in and returns to on its own.
|
||||
*/
|
||||
const box = useRef<HTMLDivElement | null>(null);
|
||||
const [following, setFollowing] = useState(true);
|
||||
|
||||
useLayoutEffect(() => {
|
||||
const node = box.current;
|
||||
if (!node || !following) return;
|
||||
node.scrollTop = node.scrollHeight;
|
||||
}, [log.length, following]);
|
||||
|
||||
// Reset to following whenever it opens, so reopening mid-run lands on the newest line rather than
|
||||
// wherever the last visit left off.
|
||||
useEffect(() => {
|
||||
if (open) setFollowing(true);
|
||||
}, [open]);
|
||||
|
||||
const transcript = log.map((line) => `${clock(line.at)} ${line.file ? `${line.file} — ` : ''}${line.text}`).join('\n');
|
||||
|
||||
return (
|
||||
<Dialog open={open} onOpenChange={(next) => !next && onClose()}>
|
||||
<DialogContent className="flex max-h-[80vh] flex-col gap-3 sm:max-w-3xl">
|
||||
<DialogHeader>
|
||||
<DialogTitle className="flex items-center gap-2 text-sm">
|
||||
{running && <Loader2 className="size-3.5 animate-spin" />}
|
||||
Sync
|
||||
<span className="font-normal text-muted-foreground">
|
||||
{running
|
||||
? state?.total
|
||||
? `${state.done} of ${state.total}${percent === null ? '' : ` · ${percent}%`}`
|
||||
: 'reading originals…'
|
||||
: state?.finishedAt
|
||||
? 'finished'
|
||||
: 'idle'}
|
||||
</span>
|
||||
</DialogTitle>
|
||||
</DialogHeader>
|
||||
|
||||
{running && (
|
||||
<div className="h-1 w-full shrink-0 overflow-hidden rounded-full bg-muted">
|
||||
{/* Indeterminate until the walk has counted: a bar that starts full and shrinks is worse
|
||||
than one that has not started. */}
|
||||
<div
|
||||
className="h-full rounded-full bg-primary transition-[width] duration-500"
|
||||
style={{ width: `${percent ?? 0}%` }}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
<div
|
||||
ref={box}
|
||||
// `onScroll` rather than a button: following is a thing somebody stops by scrolling, which is
|
||||
// what they were already doing, and resumes by scrolling back down.
|
||||
onScroll={(event) => {
|
||||
const node = event.currentTarget;
|
||||
setFollowing(node.scrollHeight - node.scrollTop - node.clientHeight < 24);
|
||||
}}
|
||||
className="min-h-[16rem] flex-1 overflow-auto rounded-md border border-border bg-muted/30 p-3 font-mono text-[11px] leading-relaxed"
|
||||
>
|
||||
{log.length === 0 ? (
|
||||
<p className="text-muted-foreground">
|
||||
Nothing yet. Press Sync to build web-ready copies of everything in originals.
|
||||
</p>
|
||||
) : (
|
||||
log.map((line) => (
|
||||
<div key={line.n} className={`flex gap-2 ${COLOURS[line.level]}`}>
|
||||
<span className="shrink-0 opacity-50">{clock(line.at)}</span>
|
||||
{/* The file is not repeated on every rung of the same file — the eye follows a column
|
||||
of identical names badly, and the interesting part is what changed. */}
|
||||
{line.file && <span className="min-w-0 shrink-0 max-w-[45%] truncate">{line.file}</span>}
|
||||
<span className="min-w-0 flex-1 break-words">{line.text}</span>
|
||||
</div>
|
||||
))
|
||||
)}
|
||||
</div>
|
||||
|
||||
{error && <p className="shrink-0 text-xs text-destructive">{error}</p>}
|
||||
|
||||
<div className="flex shrink-0 items-center justify-between gap-2">
|
||||
<Button
|
||||
size="sm"
|
||||
variant="outline"
|
||||
className="h-7 gap-1.5 text-xs"
|
||||
disabled={log.length === 0}
|
||||
onClick={() => {
|
||||
void copyToClipboard(transcript);
|
||||
flashCopied();
|
||||
}}
|
||||
>
|
||||
{copied ? <Check className="size-3" /> : <Copy className="size-3" />}
|
||||
{copied ? 'Copied' : 'Copy'}
|
||||
</Button>
|
||||
|
||||
{/* Start from in here too. Opening the modal to watch and then having to close it to press
|
||||
the button is the sort of thing that only reads as obvious to whoever built it. */}
|
||||
<Button
|
||||
size="sm"
|
||||
className="h-7 gap-1.5 text-xs"
|
||||
onClick={() => void start()}
|
||||
disabled={busy || running}
|
||||
>
|
||||
{running ? <Loader2 className="size-3 animate-spin" /> : <RefreshCw className="size-3" />}
|
||||
{running ? 'Syncing' : 'Sync'}
|
||||
</Button>
|
||||
</div>
|
||||
</DialogContent>
|
||||
</Dialog>
|
||||
);
|
||||
};
|
||||
+59
-3
@@ -31,6 +31,17 @@ import { readableError } from './errors';
|
||||
|
||||
export type SyncError = { file: string; message: string };
|
||||
|
||||
/** One line of the run's transcript. Mirrors LogLine in api/sync.ts. */
|
||||
export type LogLine = {
|
||||
/** Monotonic within a run. What `?since=` is asked with, so each line transfers once. */
|
||||
n: number;
|
||||
at: string;
|
||||
level: 'info' | 'step' | 'done' | 'warn' | 'error';
|
||||
/** The file this is about, or blank for the run's own lines. */
|
||||
file: string;
|
||||
text: string;
|
||||
};
|
||||
|
||||
export type SyncState = {
|
||||
running: boolean;
|
||||
total: number | null;
|
||||
@@ -43,6 +54,8 @@ export type SyncState = {
|
||||
pruned: number;
|
||||
errors: SyncError[];
|
||||
current: string[];
|
||||
/** Only the lines the server had not already sent — see the `since` note in the feed. */
|
||||
log: LogLine[];
|
||||
startedAt: string | null;
|
||||
finishedAt: string | null;
|
||||
fatal: string | null;
|
||||
@@ -75,12 +88,21 @@ const KEY = ['fofos-admin', 'media-sync'];
|
||||
type Shared = {
|
||||
state: SyncState | null;
|
||||
ledger: Ledger | null;
|
||||
/**
|
||||
* The transcript, ACCUMULATED — the server sends deltas and this is the whole of it.
|
||||
*
|
||||
* Kept here rather than in the modal so closing the modal does not throw away the run's history.
|
||||
* Somebody who opens it, watches, closes it and opens it again is watching one run, not two.
|
||||
*/
|
||||
log: LogLine[];
|
||||
/** Which run `log` belongs to, so a new one clears it. See the note where it is compared. */
|
||||
logRun: string | null;
|
||||
/** A start request is in flight — distinct from `state.running`, which is the SERVER's answer. */
|
||||
busy: boolean;
|
||||
error: string | null;
|
||||
};
|
||||
|
||||
const EMPTY: Shared = { state: null, ledger: null, busy: false, error: null };
|
||||
const EMPTY: Shared = { state: null, ledger: null, log: [], logRun: null, busy: false, error: null };
|
||||
|
||||
export type MediaSync = Shared & {
|
||||
/** A pass is under way. */
|
||||
@@ -147,10 +169,44 @@ export const useMediaSyncFeed = (): void => {
|
||||
const setRef = useRef(setShared);
|
||||
setRef.current = setShared;
|
||||
|
||||
// The latest value, for the poll to read WITHOUT being rebuilt when it changes — an interval whose
|
||||
// callback identity changed every line would be torn down and restarted on every line.
|
||||
const sharedRef = useRef(shared);
|
||||
sharedRef.current = shared;
|
||||
|
||||
const load = useCallback(async () => {
|
||||
const held = sharedRef.current;
|
||||
const since = held.log.at(-1)?.n;
|
||||
|
||||
try {
|
||||
const state = await clientRef.current.get<SyncState>('/fofos-admin/_config/media/sync');
|
||||
setRef.current((was) => ({ ...was, state }));
|
||||
/*
|
||||
Ask only for what we do not have.
|
||||
|
||||
A long run's transcript is thousands of lines and this polls every second, so without `since`
|
||||
the same megabyte would arrive over and over. On the first poll of a run there is nothing held,
|
||||
so `since` is absent and the server sends the lot — which is what a modal opened mid-run wants.
|
||||
*/
|
||||
const state = await clientRef.current.get<SyncState>(
|
||||
since === undefined
|
||||
? '/fofos-admin/_config/media/sync'
|
||||
: `/fofos-admin/_config/media/sync?since=${since}`,
|
||||
);
|
||||
|
||||
setRef.current((was) => {
|
||||
/*
|
||||
A NEW run resets the transcript.
|
||||
|
||||
`n` restarts at 1 each run, so appending across a boundary would interleave two runs and put
|
||||
the numbers out of order. `startedAt` is what identifies a run — comparing it is cheaper and
|
||||
more honest than watching `running` go false then true, which misses a run that began and
|
||||
ended between two polls.
|
||||
*/
|
||||
const run = state.startedAt;
|
||||
const fresh = run !== was.logRun;
|
||||
const log = fresh ? state.log : [...was.log, ...state.log];
|
||||
|
||||
return { ...was, state, log, logRun: run };
|
||||
});
|
||||
} catch {
|
||||
// Owner-only. Root Location has already said so once if that is the reason.
|
||||
setRef.current((was) => ({ ...was, state: null }));
|
||||
|
||||
Reference in New Issue
Block a user