conversations come back as conversations

A probe on the torch stage worker (~/dockers/torch-worker) runs beside every
transcription and decides what the recording is: solo dictation returns as
paragraphs exactly as before, several voices return speaker-labeled with
conversation: true. Nobody declares which — the caller cannot know.

Transcription and the worker each get a serial FIFO queue on their own
resource, so the probe costs a solo memo nothing and multi-user traffic
stays fair. The worker is optional and best-effort throughout: unreachable
or failing, everything still transcribes, just never labeled.

Speaker attribution is segment-granularity by maximal turn overlap —
whisper.cpp's own timestamps, no alignment stage. If boundaries prove
sloppy, wav2vec2 alignment returns as a worker endpoint.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
2026-08-25 04:27:14 +00:00
co-authored by Claude Fable 5
parent bd08f9a5fe
commit dd3dbb611b
10 changed files with 416 additions and 20 deletions
+15 -3
View File
@@ -6,9 +6,11 @@ import type { PluginManifest } from '@@/plugins/manifest';
* ── What this plugin is, and what it is not ──
*
* It does NOT run Whisper, and it does not ship a model. You point it at a whisper.cpp `/inference`
* server you already run (the same container the platform used to talk to directly) and it forwards to
* it. Nothing is transcribed here — the sidecar's whole job is the request shape: detect the language,
* then transcribe faithfully in it, tuned for one person talking into a phone.
* server you already run (the same container the platform used to talk to directly) and, optionally, at
* a diarization worker beside it. Nothing is transcribed here — the sidecar's job is the request shape
* (detect the language, then transcribe faithfully in it, tuned for one person talking into a phone)
* and the pipeline around it: probe whether the recording is a conversation, label the speakers when it
* is, and queue multi-user traffic fairly across the two servers.
*
* Named for the feature, not the implementation — the same reasoning as `photos` not `immich`. Swapping
* the backend for a different `/inference`-compatible server would not move the mount point or the route
@@ -58,6 +60,16 @@ export const manifest: PluginManifest = {
required: true,
placeholder: 'http://127.0.0.1:8178',
},
{
key: 'workerUrl',
label: 'Diarization worker',
hint:
'The torch stage worker that detects conversations and labels speakers. Optional — ' +
'without it, recordings still transcribe but nothing is speaker-labeled.',
type: 'url',
required: false,
placeholder: 'http://127.0.0.1:8183',
},
],
permissions: [