extract speech-to-text into its own plugin
Moves platform STT out into a plugin that owns the whole whisper.cpp contract: sidecar, config (a JSON file under DATA_PATH/transcribe/, validated against the server before storing), a thin proxy router, a skill so an agent knows the contract without being told, and a script-mode file-browser task (not agentic — no model turn per audio file). sidecar/transcribe.ts and its test moved unchanged from src/servers/api/stt/ — every constant in it has a reason attached and none of it was touched. The platform-side removal (routes, dead useAudioRecording path, Settings screen, locale keys) lands in a separate, uncommitted change in the platform repo for review. Not yet reachable from the file browser: the platform does not load tasks from a plugin's directory yet (skills/affordances/notifications do). Ships anyway per instructions. Officer-Session: 656f5c85-e49a-4f1f-b9f3-042a04b4d043
This commit is contained in:
@@ -0,0 +1,69 @@
|
||||
---
|
||||
name: Transcribe
|
||||
description: Use when you need to turn a recording into text — a voice memo, a meeting, dictation on disk. Carries the route, the request shape, and what to do when nothing is configured.
|
||||
version: 1
|
||||
---
|
||||
|
||||
Officer's speech-to-text runs entirely inside the `transcribe` plugin, over your own whisper.cpp server.
|
||||
Nothing else on the platform reads audio for you — a request that mentions a recording, a voice memo, a
|
||||
call, a lecture, means this plugin.
|
||||
|
||||
**There is no model turn here.** The plugin does the whole job itself: detect the spoken language, then
|
||||
transcribe faithfully in it. Do not narrate the audio yourself, do not summarize it in place of
|
||||
transcribing, and do not chain a translation step — the plugin is already faithful to the source language,
|
||||
and Whisper's translate mode is English-only, so anything past it would be a rough loss rather than a
|
||||
feature.
|
||||
|
||||
## Is it set up?
|
||||
|
||||
```
|
||||
GET /api/transcribe/config
|
||||
→ { "configured": true, "url": "http://192.168.1.50:64201" }
|
||||
→ { "configured": false, "url": null }
|
||||
```
|
||||
|
||||
If `configured` is `false`, say so plainly and stop — "speech-to-text isn't set up yet; point it at a
|
||||
whisper.cpp server with `PUT /api/transcribe/config`" — rather than attempting the transcription and
|
||||
reporting a generic failure. Setting it up is a one-time, deliberate act (it validates the URL against the
|
||||
server before storing it), not something to do silently on the caller's behalf without being asked.
|
||||
|
||||
```
|
||||
PUT /api/transcribe/config
|
||||
{ "url": "http://192.168.1.50:64201" }
|
||||
→ { "configured": true, "url": "http://192.168.1.50:64201" }
|
||||
→ 400 { "error": "…" } — url missing, malformed, or the server did not answer
|
||||
```
|
||||
|
||||
## Transcribing
|
||||
|
||||
```
|
||||
POST /api/transcribe/transcribe
|
||||
multipart/form-data:
|
||||
file — the audio file (mp3, wav, m4a, whatever whisper.cpp accepts)
|
||||
spokenLanguages — zero or more, e.g. spokenLanguages=en&spokenLanguages=pt
|
||||
|
||||
→ { "text": "…", "detectedLanguage": "en" }
|
||||
→ 400 { "error": "Whisper not configured — PUT /api/transcribe/config first" }
|
||||
→ 502 { "error": "…" } — the whisper.cpp server rejected or failed the request
|
||||
```
|
||||
|
||||
`spokenLanguages` only matters when Whisper's own detection is genuinely unsure — it breaks the tie
|
||||
toward a language you already know the speaker uses. Pass what you know; pass nothing and detection still
|
||||
works, it is just less informed on an ambiguous clip.
|
||||
|
||||
**This can take minutes** on a long recording — whisper.cpp runs at roughly 8x realtime, so a 40-minute
|
||||
memo is a couple of minutes of waiting, not a couple of seconds. Do not treat a slow response as a failure
|
||||
and retry; wait for it.
|
||||
|
||||
## The file-browser task
|
||||
|
||||
`tasks/transcribe-audio-file` in this plugin ships a right-click task for `.mp3`/`.wav`/`.m4a` files that
|
||||
calls this same route and files the result next to the original — a **script** task, deliberately, with no
|
||||
model turn per audio file. If you are asked to transcribe something already sitting in the file browser,
|
||||
prefer pointing the user at that over doing it yourself turn-by-turn.
|
||||
|
||||
## What this is not
|
||||
|
||||
Not a live dictation microphone — there is no streaming endpoint, only a file upload. Not a translator —
|
||||
it transcribes in whatever language was spoken, never into a different one. And it holds no memory of past
|
||||
transcriptions; every call is independent.
|
||||
Reference in New Issue
Block a user