pastilhasandClaude Fable 5
dd3dbb611b
A probe on the torch stage worker (~/dockers/torch-worker) runs beside every transcription and decides what the recording is: solo dictation returns as paragraphs exactly as before, several voices return speaker-labeled with conversation: true. Nobody declares which — the caller cannot know. Transcription and the worker each get a serial FIFO queue on their own resource, so the probe costs a solo memo nothing and multi-user traffic stays fair. The worker is optional and best-effort throughout: unreachable or failing, everything still transcribes, just never labeled. Speaker attribution is segment-granularity by maximal turn overlap — whisper.cpp's own timestamps, no alignment stage. If boundaries prove sloppy, wav2vec2 alignment returns as a worker endpoint. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
83 lines
3.3 KiB
TypeScript
83 lines
3.3 KiB
TypeScript
import type { PluginManifest } from '@@/plugins/manifest';
|
|
|
|
/**
|
|
* Transcribe — Officer's speech-to-text, talking to a self-hosted whisper.cpp server.
|
|
*
|
|
* ── What this plugin is, and what it is not ──
|
|
*
|
|
* It does NOT run Whisper, and it does not ship a model. You point it at a whisper.cpp `/inference`
|
|
* server you already run (the same container the platform used to talk to directly) and, optionally, at
|
|
* a diarization worker beside it. Nothing is transcribed here — the sidecar's job is the request shape
|
|
* (detect the language, then transcribe faithfully in it, tuned for one person talking into a phone)
|
|
* and the pipeline around it: probe whether the recording is a conversation, label the speakers when it
|
|
* is, and queue multi-user traffic fairly across the two servers.
|
|
*
|
|
* Named for the feature, not the implementation — the same reasoning as `photos` not `immich`. Swapping
|
|
* the backend for a different `/inference`-compatible server would not move the mount point or the route
|
|
* shape.
|
|
*
|
|
* ── Where the connection lives ──
|
|
*
|
|
* A single URL, in a JSON file under `DATA_PATH/transcribe/`, not in Postgres and not in this
|
|
* repository. There is only ever one Whisper server for the whole install — unlike `photos_config` or
|
|
* `jellyfin_servers`, nobody has a reason to keep more than one — so a file the owner can also edit by
|
|
* hand is simpler than a table with one row. See `sidecar/config.ts`.
|
|
*
|
|
* ── No `osDependencies` ──
|
|
*
|
|
* Nothing shells out. The sidecar reaches whisper.cpp with `fetch`, the same as the code it replaces did.
|
|
*
|
|
* ── No `web/`, and no settings screen of its own ──
|
|
*
|
|
* This plugin ships no dashboard route and no dock tile — `plugin.web` gates the tile, and a
|
|
* backend-only plugin has nothing to pin one to. It is reached from a task's right-click menu and by an
|
|
* agent that has read its skill, not by a screen.
|
|
*/
|
|
export const manifest: PluginManifest = {
|
|
publisher: 'officerdev',
|
|
version: '1.0.0',
|
|
platform: '>=1.0.0',
|
|
|
|
label: 'Transcribe',
|
|
summary: 'Speech-to-text over your own whisper.cpp server',
|
|
icon: 'AudioLines',
|
|
color: '#f97316',
|
|
|
|
/**
|
|
* The whisper.cpp server this talks to. Declared rather than rendered: this plugin ships no `web/`, so
|
|
* without a declaration there is nowhere to set it and the only way in is an HTTP call somebody has to
|
|
* know about — which is the state it shipped in.
|
|
*
|
|
* The platform renders the form and PUTs to this plugin's own `/config`. It stores nothing: the URL
|
|
* lives where this plugin keeps it, and validation against the server still happens here.
|
|
*/
|
|
settings: [
|
|
{
|
|
key: 'url',
|
|
label: 'Whisper server',
|
|
hint: 'The whisper.cpp server to transcribe through. Checked before it is saved.',
|
|
type: 'url',
|
|
required: true,
|
|
placeholder: 'http://127.0.0.1:8178',
|
|
},
|
|
{
|
|
key: 'workerUrl',
|
|
label: 'Diarization worker',
|
|
hint:
|
|
'The torch stage worker that detects conversations and labels speakers. Optional — ' +
|
|
'without it, recordings still transcribe but nothing is speaker-labeled.',
|
|
type: 'url',
|
|
required: false,
|
|
placeholder: 'http://127.0.0.1:8183',
|
|
},
|
|
],
|
|
|
|
permissions: [
|
|
{
|
|
key: 'transcribe',
|
|
label: 'Transcribe',
|
|
description: 'Turn audio into text over your own whisper.cpp server',
|
|
},
|
|
],
|
|
};
|