Files
transcribe/PLUGIN.md
T
pastilhas 3da6d11c00 extract speech-to-text into its own plugin
Moves platform STT out into a plugin that owns the whole whisper.cpp contract: sidecar,
config (a JSON file under DATA_PATH/transcribe/, validated against the server before
storing), a thin proxy router, a skill so an agent knows the contract without being told,
and a script-mode file-browser task (not agentic — no model turn per audio file).

sidecar/transcribe.ts and its test moved unchanged from src/servers/api/stt/ — every
constant in it has a reason attached and none of it was touched.

The platform-side removal (routes, dead useAudioRecording path, Settings screen, locale
keys) lands in a separate, uncommitted change in the platform repo for review.

Not yet reachable from the file browser: the platform does not load tasks from a plugin's
directory yet (skills/affordances/notifications do). Ships anyway per instructions.

Officer-Session: 656f5c85-e49a-4f1f-b9f3-042a04b4d043
2026-08-24 03:28:04 +00:00

5.0 KiB

Transcribe — the extraction record

Extracted from the platform on 2026-08-24. plugins/EXTRACTING-A-PLUGIN.md is the runbook; this file is what it cost.

What moved

Was Is now
src/servers/api/stt/transcribe.ts sidecar/transcribe.ts — unchanged
src/servers/api/stt/transcribe.test.ts sidecar/transcribe.test.ts — unchanged
src/servers/api/server-settings/stt.ts sidecar/config.ts — rewritten

sidecar/transcribe.ts is the reason this extraction exists at all: 416 lines of tuning — the two-pass detect-then-transcribe, 25 language priming prompts, the VAD widening, the paragraphing — each with a reason in its own comment, and none of it touched. Moving it, not rewriting it, was the instruction and the only defensible one: every constant in that file was chosen against a real recording at some point, and there is no way to re-derive that from reading the code cold.

What did not move unchanged: config

The platform kept a single admin-only setting, server-settings/stt — one row in a JSON blob (readServerSettings/writeServerSettings), gated by the server-admin permission and reachable only by the owner. That shape does not exist here to move into: a plugin's config is its own file, not a slot in the platform's settings blob, and this plugin's own permission (transcribe) is grantable to anyone, not owner-only.

So sidecar/config.ts is new, not moved: a JSON file under DATA_PATH/transcribe/, following the DATA_PATH/<plugin>/ convention plugins/music/sidecar/indexer.ts established, with the same validate-before-store behaviour the old /server-settings/stt/test route had (an empty multipart POST to /inference — Whisper always rejects it, but any response at all means the server exists).

Deliberately not owner-only. Photos' /_config holds a real credential (an Immich API key) and is still reachable by any account holding the permission at write; jellyfin's manifest, despite the ownerOnly field existing for exactly this shape, does not set it either. A Whisper URL is not a credential — there is nothing to steal by reading it, and nothing more dangerous than "the transcriptions go to a different server" by changing it — so this follows the same precedent rather than inventing a narrower one.

What the platform lost

Deleted, not commented — see the platform-side diff for the full list: src/servers/api/stt/, the server-settings/stt router and its mount, POST /chat/stt, the dead useAudioRecording STT path (blobToWav stays; DictateDialog.tsx still needs it for something that was never STT — see below), POST /file-browser/transcribe and the FileViewer handler behind the button that was already hidden, the Settings → Speech to Text screen, and the orphaned locale keys across all six officer-web locales.

One thing found and left alone. DictateDialog.tsx in the file browser imports blobToWav from the same file useAudioRecording's dead STT path lived in. It records audio and uploads the raw .wav — it never called Whisper, was never part of this feature, and stays in the platform exactly as it is. Deleting the whole file would have taken it down as collateral.

The task: script, not agentic

The worked reference this shipped against (~/dockers/officer.dev/seed-archive/tasks/transcribe-audio-file/ on alpha) is agentic: it runs its own detect-language pass before transcribing, and generates an archive-folder title from the transcription's contents. Both are wrong for a script task and were dropped on purpose rather than carried over — the detect step duplicates what the plugin's route already does server-side, and a generated title needs a model turn, which is the entire thing "script, not agentic" was asked for to avoid spending per file. The filing behaviour (a dated, slugged sibling directory, original moved in, transcript saved beside it) is the part worth keeping, so it survived; the slug now comes from the original filename instead of a generated title.

What is NOT wired up yet

The platform does not load tasks from a plugin's directory. Skills, affordances and notifications are read from plugins/<name>/ at runtime (docs/plugin-capabilities.md); tasks are not — task-files.ts only reads $OFFICER_ITEMS_DIR/tasks/. So tasks/transcribe-audio-file/ ships here, is correct, and will not appear in the file browser's right-click menu until that loader lands. Not a defect in this plugin; recorded so nobody spends an evening looking for why it is missing.

Still open

  • No assets/icon.png. Falls back to the AudioLines lucide glyph.
  • Single Whisper server, not a registry. Unlike photos_config or jellyfin_servers, there is exactly one URL and no way to keep a second on hand. Revisit if that ever turns out to be wanted — nothing here forecloses it, but nothing asked for it either.