a plugin can claim a file type the platform cannot read

the file browser's thumbnailer read what sharp and ffmpeg read, and that list was a
constant in core. HEIC forced the question: sharp's bundled libheif has no HEVC
decoder, and teaching core to shell out to ImageMagick means the PLATFORM acquiring
a system dependency for one format. that is a plugin's job.

so a manifest may declare `fileTypes` — extensions, plus a thumbnail path and a
render path on its own sidecar. `ls` unions those with core's, so a file only
promises a thumbnail when something can actually produce one. `/thumb` forwards a
claimed type to whoever claimed it.

CORE DOES NOT KNOW av-club EXISTS. it reads a claim at runtime from whatever is
installed and enabled, calls a path by name, and falls back to a file icon when
nothing claims the type. no import, no name, no special case.

PROVEN BY DISABLING IT: heic reports thumb=image with av-club enabled, thumb=none
disabled, image again re-enabled. that is the whole requirement — the file browser
is what it was without the plugin — and it is a test rather than a claim.

the plugin's bytes are cached by CORE, under the same key everything else uses, so
a HEIC thumbnail is produced once and then served from disk like any other. that is
why `buildThumbnail` takes a producer rather than there being a second path beside
it: the cache key with its mtime and size, the concurrency cap and the remembered
failures all live in that one function, and a parallel path would be a second place
to forget the negative caching.

TWO THINGS FOUND BY WIRING IT UP. a claimed-but-not-running sidecar is skipped
rather than recorded, or `ls` promises a thumbnail that cannot be fetched while a
plugin restarts. and the sidecar authenticates: it accepts `x-officer-user` from
anything on loopback, by its own design, so core sends the account the request is
FOR — passing anything else would be lying to a plugin that scopes per user.

verified: HEIC 2 MB in, 12 KB out, 5.4s cold and 101ms cached; png and mp4
unchanged; 1665 tests; docs:check green.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Officer-Session: 656f5c85-e49a-4f1f-b9f3-042a04b4d043

src/servers/api/file-browser/plugin-file-types.ts

@@ -0,0 +1,99 @@
+import { snapshotPlugins } from '../../plugins/mount';
+import { sidecarPort } from '../../sidecar-registry';
+
+/**
+ * File types an installed plugin can show and the platform cannot.
+ *
+ * ── One place, so three consumers cannot disagree ──
+ *
+ * `ls` decides whether an entry promises a thumbnail, `/thumb` decides where to get one, and the viewer
+ * decides whether a full-size render exists. Those must be the same answer: a listing that promises a
+ * thumbnail nothing will produce is a request that 404s on every scroll, and a viewer that offers a
+ * render for a type nothing claims is a spinner that never resolves.
+ *
+ * ── Core does not know which plugin this is ──
+ *
+ * It reads `fileTypes` from whatever is installed AND ENABLED, and calls a path by name on that
+ * plugin's own sidecar. Nothing here imports plugin code and nothing names av-club. Disable the plugin
+ * and the claims vanish with it, which is what makes the file browser degrade to exactly what it was
+ * rather than to something broken.
+ *
+ * ── Why the sidecar directly, and not our own HTTP surface ──
+ *
+ * A loopback request back through `/api` would need a credential to satisfy the platform's own auth,
+ * and minting one so the server can call itself is a worse thing to have than this. `sidecarPort` is
+ * the port the plugin announced when it started; the proxy in front of it exists for browsers, and the
+ * server is already inside.
+ *
+ * ── `x-officer-user` is the whole authentication, and that is the sidecar's own design ──
+ *
+ * A plugin sidecar binds loopback and accepts that header from anything already on the machine — the
+ * platform is the thing that decides who may call, and by the time this runs the account gate and the
+ * permission gate have both passed. `sidecarFetch` below sends it because the proxy in front of the
+ * sidecar sends it, and a caller that skips it gets a 401 that looks like the plugin being broken.
+ */
+
+export type FileTypeClaim = {
+  plugin: string;
+  /** Absolute URL on the plugin's sidecar, or null when it did not offer that half. */
+  thumbnail: string | null;
+  render: string | null;
+};
+
+/**
+ * Extension → who claims it. Lowercase keys, no leading dot.
+ *
+ * Not cached: `snapshotPlugins` reads the database, and this is called on a listing that is already
+ * doing database work. A cache would need invalidating on install, uninstall, enable and disable —
+ * four events to get wrong in exchange for a lookup that is already cheap.
+ */
+export async function pluginFileTypes(): Promise<Map<string, FileTypeClaim>> {
+  const claims = new Map<string, FileTypeClaim>();
+
+  let states: Awaited<ReturnType<typeof snapshotPlugins>>['states'] = [];
+  try {
+    ({ states } = await snapshotPlugins());
+  } catch {
+    // The file browser still works. A database that cannot be read is a much louder problem elsewhere,
+    // and losing every thumbnail to it would only obscure that.
+    return claims;
+  }
+
+  for (const state of states) {
+    if (!state.install?.enabled) continue;
+
+    const declared = state.plugin.manifest?.fileTypes;
+    if (!declared?.extensions?.length) continue;
+
+    const port = sidecarPort(state.plugin.appName);
+    // Claimed but not running. Skipped rather than recorded, so `ls` does not promise a thumbnail that
+    // cannot be fetched while the sidecar is restarting.
+    if (!port) continue;
+
+    const base = `http://127.0.0.1:${port}`;
+    for (const raw of declared.extensions) {
+      const ext = String(raw).toLowerCase().replace(/^\./, '');
+      // First claim wins. Two plugins claiming one extension is a conflict nobody has yet, and picking
+      // arbitrarily-but-stably beats picking differently on each request.
+      if (!ext || claims.has(ext)) continue;
+
+      claims.set(ext, {
+        plugin: state.plugin.appName,
+        thumbnail: declared.thumbnail ? `${base}${declared.thumbnail}` : null,
+        render: declared.render ? `${base}${declared.render}` : null,
+      });
+    }
+  }
+
+  return claims;
+}
+
+/**
+ * Call a plugin's sidecar the way its proxy would.
+ *
+ * The email is the account the request is being made FOR, not the server — a plugin scoping anything
+ * per user reads this, and passing something else would be lying to it about who is asking.
+ */
+export async function sidecarFetch(url: string, email: string): Promise<Response> {
+  return fetch(url, { headers: { 'x-officer-user': email } });
+}

src/servers/api/file-browser/router.ts

@@ -10,6 +10,7 @@ import { readTtsConfig } from '@@/api/server-settings/tts';
 import { readOcrConfig } from '@@/api/server-settings/ocr';
 import { getUserSettings } from 'officerdb';
 import { buildThumbnail, clampSize, thumbnailKind } from './thumbnails';
+import { pluginFileTypes, sidecarFetch } from './plugin-file-types';
 
 async function getUserTtsVoice(userId: number): Promise<string | null> {
   try {
@@ -204,6 +205,12 @@ router.get('/ls', async (ctx) => {
     }
     return ctx.json({ path: '/', entries: [], reset: true });
   }
+  /*
+    What a plugin can show that we cannot. Resolved ONCE per listing rather than per entry — it reads the
+    database, and a folder of two hundred files would otherwise ask two hundred times for one answer.
+  */
+  const claimed = await pluginFileTypes();
+
   const entries = await Promise.all(
     names.map(async (name) => {
       const fullPath = resolve(absPath, name);
@@ -228,7 +235,18 @@ router.get('/ls', async (ctx) => {
 
           Costs nothing: this entry has just been stat'd and the answer is a lookup on its name.
         */
-        thumb: isDirectory ? null : thumbnailKind(name),
+        /*
+          Core's own answer first, then a plugin's claim.
+
+          A file only says `thumb` when something can actually produce one: core for the formats sharp
+          and ffmpeg read, a plugin for what it declared in `fileTypes`. Without that second half, `.heic`
+          promised a thumbnail nothing would make and cost a 422 on every scroll — which is why core
+          stopped claiming it.
+        */
+        thumb: isDirectory
+          ? null
+          : (thumbnailKind(name) ??
+            (claimed.has(name.slice(name.lastIndexOf('.') + 1).toLowerCase()) ? 'image' : null)),
       };
     }),
   );
@@ -375,12 +393,42 @@ router.get('/thumb', async (ctx) => {
   if (!relPath) throw errors.BAD_REQUEST('path is required');
 
   const absPath = resolveUserPath(rootDir, relPath);
+  const size = clampSize(ctx.req.query('size'));
   const kind = thumbnailKind(relPath);
-  // 415 rather than 404: the file is there, it is simply not something a frame can be taken from. The
-  // client does not ask for these, so this is the answer to a stale tab rather than to normal use.
-  if (!kind) return ctx.text('no thumbnail for this file type', 415);
 
-  const result = await buildThumbnail(absPath, getUserDataDir(user.email), kind, clampSize(ctx.req.query('size')));
+  /*
+    A type core cannot read, but an installed plugin can.
+
+    Forwarded to the plugin's own sidecar and cached HERE, under the same key everything else uses —
+    so a HEIC thumbnail is produced once and then served from disk like any other, and the plugin is
+    not asked again on every scroll.
+
+    `buildThumbnail` is given the fetch rather than a path: the caching, the concurrency limit and the
+    remembered failures are all in that one function, and a second copy of them here for this one case
+    is how the two would come to behave differently.
+  */
+  if (!kind) {
+    const claimed = (await pluginFileTypes()).get(relPath.slice(relPath.lastIndexOf('.') + 1).toLowerCase());
+    if (claimed?.thumbnail) {
+      const fetched = await buildThumbnail(absPath, getUserDataDir(user.email), 'image', size, async () => {
+        const res = await sidecarFetch(
+          `${claimed.thumbnail}?path=${encodeURIComponent(absPath)}&size=${size}`,
+          user.email,
+        );
+        return res.ok ? Buffer.from(await res.arrayBuffer()) : null;
+      });
+      if (!fetched.ok) return ctx.text(fetched.reason, 422);
+      return new Response(Bun.file(fetched.path), {
+        headers: { 'Content-Type': 'image/jpeg', 'Cache-Control': 'private, max-age=31536000, immutable' },
+      });
+    }
+
+    // 415 rather than 404: the file is there, it is simply not something anything here can read. The
+    // client does not ask for these, so this answers a stale tab rather than normal use.
+    return ctx.text('no thumbnail for this file type', 415);
+  }
+
+  const result = await buildThumbnail(absPath, getUserDataDir(user.email), kind, size);
   // 422, and the client falls back to the file icon. Not 500: a file ffmpeg cannot read is a fact about
   // the file, and logging it as a server error would bury the ones that are.
   if (!result.ok) return ctx.text(result.reason, 422);

src/servers/api/file-browser/thumbnails.ts

@@ -216,6 +216,16 @@ export async function buildThumbnail(
   userDataDir: string,
   kind: 'image' | 'video',
   size: number,
+  /**
+   * Produce the bytes yourself instead of letting sharp or ffmpeg try.
+   *
+   * For a type core cannot read but an installed plugin can — see `plugin-file-types.ts`. It is passed
+   * IN rather than handled by a second function so that everything around it is shared: the cache key
+   * with its mtime and size, the concurrency limit, and the remembered failures. A parallel path for
+   * plugin thumbnails would be a second cache to keep correct and a second place to forget the
+   * negative caching, which is the part that stops one bad file spawning work on every scroll.
+   */
+  produce?: () => Promise<Buffer | null>,
 ): Promise<ThumbnailResult> {
   const info = await stat(absPath).catch(() => null);
   if (!info || !info.isFile()) return { ok: false, reason: 'not a file' };
@@ -247,7 +257,11 @@ export async function buildThumbnail(
     const common = ['-v', 'error', '-y', '-frames:v', '1', '-vf', scale, '-f', 'mjpeg', cachePath];
 
     let ok: boolean;
-    if (kind === 'image') {
+    if (produce) {
+      const bytes = await produce().catch(() => null);
+      if (bytes?.length) await Bun.write(cachePath, bytes);
+      ok = !!bytes?.length;
+    } else if (kind === 'image') {
       ok = await resizeImage(absPath, cachePath, size);
     } else {
       // `-ss` BEFORE `-i` is the fast seek — it jumps rather than decoding a second of video first.

src/servers/plugins/manifest.ts

@@ -477,6 +477,38 @@ export type PluginManifest = {
   settings?: PluginSetting[];
 
   websocket?: { sidecar: string };
+  /**
+   * File types this plugin can show that the platform cannot.
+   *
+   * ── Why this exists ──
+   *
+   * The file browser's thumbnailer reads what sharp and ffmpeg read, and that list is a constant in
+   * core. HEIC is the case that forced the question: sharp's bundled libheif has no HEVC decoder, and
+   * teaching core to shell out to ImageMagick would mean the platform acquiring a system dependency for
+   * one format — which is a plugin's job, not core's.
+   *
+   * So a plugin may CLAIM extensions. `ls` unions these with core's own, so a file only promises a
+   * thumbnail when something can actually produce one; `/thumb` forwards a claimed type to the plugin
+   * that claimed it; and the viewer asks for `render` when opening one full size.
+   *
+   * ── The dependency still points the right way ──
+   *
+   * Core does not import plugin code and does not know av-club exists. It reads a claim at runtime from
+   * whatever is installed and enabled, calls a route by name, and falls back to a file icon when nothing
+   * claims the type. Uninstall the plugin and the file browser is exactly what it was.
+   *
+   * `thumbnail` and `render` are paths on the plugin's own sidecar, called with `?path=<absolute>` —
+   * plus `&size=` for a thumbnail and `&width=` for a render. Both must answer image bytes.
+   */
+  fileTypes?: {
+    /** Lowercase, no leading dot. `['heic', 'heif']`. */
+    extensions: string[];
+    /** Sidecar path returning a small image. Omit and the platform will not offer thumbnails for these. */
+    thumbnail?: string;
+    /** Sidecar path returning a full-size displayable image. Omit and the viewer will not offer them. */
+    render?: string;
+  };
+
   osDependencies?: OsDependency[];
 
   /**