watch originals, because a move is not an event
You should not have had to press Sync. Nine files moved into a folder and nothing noticed. Two gaps, and neither would have caught it. The platform's event bus emits `files:uploaded` and `archive:extracted` — things Officer's own file browser DID — so a move emits nothing at all, and neither would an rsync, a drop over SMB, or anything done in a terminal. And no plugin subscribes to hooks anyway. That is not a bug in the bus: it reports actions, and this needs to know about a directory. So a filesystem watcher, the same shape as the mailbox one already here. The debounce is twenty seconds and long on purpose. A move of nine files is dozens of events and an upload of two hundred is thousands, while a sync walks the whole tree — so waiting for quiet means one run after the dust settles rather than one per file. Being slightly late costs nothing, because the sync skips everything already current; being eager costs a full walk per event. It cannot feed itself: only originals/ is watched and the sync writes only to derived/. Worth stating, because the opposite arrangement is a loop that looks like a busy machine and nothing else. A run already in flight is left alone rather than queued — it walks the tree when it starts, and whatever lands after that is what the next quiet stretch is for. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,90 @@
|
||||
import { watch, type FSWatcher } from 'node:fs';
|
||||
import { mkdir } from 'node:fs/promises';
|
||||
import { db } from 'officerdb/db';
|
||||
import { getOwnerUser } from 'officerdb';
|
||||
import { fofosAdminSettings } from '../db/schema';
|
||||
import { originalsDir } from './derive';
|
||||
import { readSync, startSync } from './sync';
|
||||
|
||||
/**
|
||||
* Notice when `originals/` changes, and convert what arrived.
|
||||
*
|
||||
* ── Why a filesystem watcher and not the platform's event bus ──
|
||||
*
|
||||
* The bus was the obvious answer and it is not sufficient. It emits `files:uploaded` and
|
||||
* `archive:extracted` — things Officer's own file browser DID. Moving files into a folder emits
|
||||
* nothing at all, and neither would an rsync, a drop over SMB, or anything a person does in a terminal.
|
||||
*
|
||||
* That is not a bug in the bus: it reports actions, and this needs to know about a directory. Nine
|
||||
* files moved into a new folder produced no event, and the conversion sat waiting for somebody to
|
||||
* press a button they had no reason to think was needed.
|
||||
*
|
||||
* ── The debounce is long on purpose ──
|
||||
*
|
||||
* A move of nine files is dozens of events, an upload of two hundred is thousands, and a sync walks
|
||||
* the whole tree. Waiting for a stretch of quiet means one sync after the dust settles rather than one
|
||||
* per file — and because the sync skips everything already current, being slightly late costs nothing
|
||||
* while being eager costs a full walk per event.
|
||||
*
|
||||
* ── It cannot feed itself ──
|
||||
*
|
||||
* Only `originals/` is watched. The sync writes exclusively to `derived/`, so nothing it does can
|
||||
* trigger it again. That is worth stating because the opposite arrangement is a loop that looks like a
|
||||
* busy machine and nothing else.
|
||||
*/
|
||||
|
||||
/** Long enough that a folder of files settles into one run, short enough to feel automatic. */
|
||||
const SETTLE_MS = 20_000;
|
||||
|
||||
let watcher: FSWatcher | null = null;
|
||||
let settle: ReturnType<typeof setTimeout> | null = null;
|
||||
|
||||
async function syncIfIdle(email: string): Promise<void> {
|
||||
// A run already going will pick up whatever landed — it walks the tree when it starts, and anything
|
||||
// that arrived after that is what the next quiet stretch is for.
|
||||
if (readSync(email).running) return;
|
||||
startSync(email, { trigger: 'watcher' });
|
||||
}
|
||||
|
||||
/**
|
||||
* Start, restart or stop the watch to match what is stored.
|
||||
*
|
||||
* The DIRECTORY is watched recursively rather than each file: files are the thing that appear and
|
||||
* disappear, so a watch on one is a watch on something that may not exist yet.
|
||||
*/
|
||||
export async function watchOriginals(): Promise<void> {
|
||||
watcher?.close();
|
||||
watcher = null;
|
||||
if (settle) clearTimeout(settle);
|
||||
settle = null;
|
||||
|
||||
const row = (await db.select().from(fofosAdminSettings).limit(1))[0] ?? null;
|
||||
if (!row?.rootPath) return;
|
||||
|
||||
/*
|
||||
Whose sync this is: the owner's.
|
||||
|
||||
The media lives under one root regardless of who is looking, and every path this plugin resolves
|
||||
goes through `getOwnerHomeDir`, which returns the real login home on this installation whatever
|
||||
address it is handed. So the owner is not a choice here — it is the only account the files belong
|
||||
to, and asking the database is better than assuming an address.
|
||||
*/
|
||||
const owner = await getOwnerUser();
|
||||
if (!owner?.email) return;
|
||||
const email = owner.email;
|
||||
|
||||
const dir = await originalsDir(email);
|
||||
await mkdir(dir, { recursive: true });
|
||||
|
||||
try {
|
||||
watcher = watch(dir, { recursive: true }, () => {
|
||||
if (settle) clearTimeout(settle);
|
||||
settle = setTimeout(() => void syncIfIdle(email).catch(() => {}), SETTLE_MS);
|
||||
});
|
||||
} catch {
|
||||
// No directory, no permission, or a platform without recursive watching. Left unwatched rather
|
||||
// than throwing: this runs at import, and the plugin failing to load over a missing folder would
|
||||
// take down the panel somebody would use to fix it.
|
||||
watcher = null;
|
||||
}
|
||||
}
|
||||
@@ -11,6 +11,7 @@ import { reconcile } from './derive';
|
||||
import { createDatabases, pushSchema, readDatabases } from './databases';
|
||||
import { readBackups, restoreBackup, runBackup, saveBackups, schedule, testDestination } from './backups';
|
||||
import { watchMailbox } from './messages';
|
||||
import { watchOriginals } from './originals-watch';
|
||||
import { readServers, savePorts, siteSecrets, startServer, stopServer } from './servers';
|
||||
import { buildSite, readStatic, startStatic, stopStatic } from './static-site';
|
||||
import { join } from 'node:path';
|
||||
@@ -621,3 +622,13 @@ void schedule().catch(() => {});
|
||||
same reason as above.
|
||||
*/
|
||||
void watchMailbox().catch(() => {});
|
||||
|
||||
/*
|
||||
Watch `originals/` and convert what arrives.
|
||||
|
||||
A filesystem watcher rather than the platform's event bus, because the bus reports ACTIONS Officer
|
||||
took — an upload, an extraction — and moving files into a folder is not one of them. Nine files moved
|
||||
produced no event at all, and the conversion waited for somebody to press a button they had no reason
|
||||
to know about.
|
||||
*/
|
||||
void watchOriginals().catch(() => {});
|
||||
|
||||
Reference in New Issue
Block a user