extract speech-to-text into its own plugin

Moves platform STT out into a plugin that owns the whole whisper.cpp contract: sidecar,
config (a JSON file under DATA_PATH/transcribe/, validated against the server before
storing), a thin proxy router, a skill so an agent knows the contract without being told,
and a script-mode file-browser task (not agentic — no model turn per audio file).

sidecar/transcribe.ts and its test moved unchanged from src/servers/api/stt/ — every
constant in it has a reason attached and none of it was touched.

The platform-side removal (routes, dead useAudioRecording path, Settings screen, locale
keys) lands in a separate, uncommitted change in the platform repo for review.

Not yet reachable from the file browser: the platform does not load tasks from a plugin's
directory yet (skills/affordances/notifications do). Ships anyway per instructions.

Officer-Session: 656f5c85-e49a-4f1f-b9f3-042a04b4d043
This commit is contained in:
2026-08-24 03:28:04 +00:00
commit 3da6d11c00
11 changed files with 1192 additions and 0 deletions
+77
View File
@@ -0,0 +1,77 @@
# Transcribe — the extraction record
Extracted from the platform on **2026-08-24**. `plugins/EXTRACTING-A-PLUGIN.md` is the runbook; this
file is what it cost.
## What moved
| Was | Is now |
| --------------------------------------------- | ---------------------------------------- |
| `src/servers/api/stt/transcribe.ts` | `sidecar/transcribe.ts` — unchanged |
| `src/servers/api/stt/transcribe.test.ts` | `sidecar/transcribe.test.ts` — unchanged |
| `src/servers/api/server-settings/stt.ts` | `sidecar/config.ts` — rewritten |
`sidecar/transcribe.ts` is the reason this extraction exists at all: 416 lines of tuning — the two-pass
detect-then-transcribe, 25 language priming prompts, the VAD widening, the paragraphing — each with a
reason in its own comment, and none of it touched. Moving it, not rewriting it, was the instruction and
the only defensible one: every constant in that file was chosen against a real recording at some point,
and there is no way to re-derive that from reading the code cold.
## What did not move unchanged: config
The platform kept a single admin-only setting, `server-settings/stt` — one row in a JSON blob
(`readServerSettings`/`writeServerSettings`), gated by the `server-admin` permission and reachable only
by the owner. That shape does not exist here to move into: a plugin's config is its own file, not a slot
in the platform's settings blob, and this plugin's own permission (`transcribe`) is grantable to anyone,
not owner-only.
So `sidecar/config.ts` is new, not moved: a JSON file under `DATA_PATH/transcribe/`, following the
`DATA_PATH/<plugin>/` convention `plugins/music/sidecar/indexer.ts` established, with the same
validate-before-store behaviour the old `/server-settings/stt/test` route had (an empty multipart POST to
`/inference` — Whisper always rejects it, but any response at all means the server exists).
**Deliberately not owner-only.** Photos' `/_config` holds a real credential (an Immich API key) and is
still reachable by any account holding the permission at write; jellyfin's manifest, despite the
`ownerOnly` field existing for exactly this shape, does not set it either. A Whisper URL is not a
credential — there is nothing to steal by reading it, and nothing more dangerous than "the transcriptions
go to a different server" by changing it — so this follows the same precedent rather than inventing a
narrower one.
## What the platform lost
Deleted, not commented — see the platform-side diff for the full list: `src/servers/api/stt/`, the
`server-settings/stt` router and its mount, `POST /chat/stt`, the dead `useAudioRecording` STT path
(`blobToWav` stays; `DictateDialog.tsx` still needs it for something that was never STT — see below),
`POST /file-browser/transcribe` and the FileViewer handler behind the button that was already hidden, the
Settings → Speech to Text screen, and the orphaned locale keys across all six `officer-web` locales.
**One thing found and left alone.** `DictateDialog.tsx` in the file browser imports `blobToWav` from the
same file `useAudioRecording`'s dead STT path lived in. It records audio and uploads the raw `.wav` — it
never called Whisper, was never part of this feature, and stays in the platform exactly as it is. Deleting
the whole file would have taken it down as collateral.
## The task: script, not agentic
The worked reference this shipped against (`~/dockers/officer.dev/seed-archive/tasks/transcribe-audio-file/`
on alpha) is agentic: it runs its own detect-language pass before transcribing, and generates an
archive-folder title from the transcription's contents. Both are wrong for a script task and were dropped
on purpose rather than carried over — the detect step duplicates what the plugin's route already does
server-side, and a generated title needs a model turn, which is the entire thing "script, not agentic" was
asked for to avoid spending per file. The filing behaviour (a dated, slugged sibling directory, original
moved in, transcript saved beside it) is the part worth keeping, so it survived; the slug now comes from
the original filename instead of a generated title.
## What is NOT wired up yet
**The platform does not load tasks from a plugin's directory.** Skills, affordances and notifications are
read from `plugins/<name>/` at runtime (`docs/plugin-capabilities.md`); tasks are not — `task-files.ts`
only reads `$OFFICER_ITEMS_DIR/tasks/`. So `tasks/transcribe-audio-file/` ships here, is correct, and will
not appear in the file browser's right-click menu until that loader lands. Not a defect in this plugin;
recorded so nobody spends an evening looking for why it is missing.
## Still open
- **No `assets/icon.png`.** Falls back to the `AudioLines` lucide glyph.
- **Single Whisper server, not a registry.** Unlike `photos_config` or `jellyfin_servers`, there is
exactly one URL and no way to keep a second on hand. Revisit if that ever turns out to be wanted —
nothing here forecloses it, but nothing asked for it either.