Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e0d80fd3ae | ||
|
|
a10757ba7c | ||
|
|
e35ff5c7ad | ||
|
|
9ddd23c388 |
@@ -0,0 +1,74 @@
|
|||||||
|
# 2026.08.24.01 — deploy report
|
||||||
|
|
||||||
|
**5 of 5 target hosts succeeded.** Two hosts skipped by plan, neither a failure.
|
||||||
|
|
||||||
|
Every host went `2646842c` → `c813eef6`, the tag commit, by fast-forward. No host was dirty in tracked
|
||||||
|
files. No `bun install` was run — the plan forbade it and `bun.lock` was untouched. No direct SQL was run
|
||||||
|
and none was authorised.
|
||||||
|
|
||||||
|
| Host | Before → after | db:push | plugins | `officer` | Result |
|
||||||
|
| --- | --- | --- | --- | --- | --- |
|
||||||
|
| `alpha` | `2646842c` → `c813eef6` | applied 2 statements (see below) | 13/13 | online, 73 → 74 | ✅ |
|
||||||
|
| `fofos` | `2646842c` → `c813eef6` | applied 2 statements | 1/1 (gitea) | online, 26 → 27 | ✅ |
|
||||||
|
| `production-pertento-bridge` | `2646842c` → `c813eef6` | applied 2 statements | 1/1 (gitea) | online, 5 → 6 | ✅ |
|
||||||
|
| `deadpool-rezio-bridge` | `2646842c` → `c813eef6` | applied 2 statements | 2/2 (email, offscale) | online, 11 → 12 | ✅ |
|
||||||
|
| `macbookpro` | `2646842c` → `c813eef6` | applied 2 statements | none installed | online, 13 → 14 | ✅ |
|
||||||
|
| `officer-dev` | `c813eef6` (already) | not run | not run | **not restarted** | ⏭ skipped — self |
|
||||||
|
| `edge-pertento-bridge` | — | — | — | — | ⏭ skipped — `docs/servers.md` |
|
||||||
|
|
||||||
|
Every process on every host reported `online` after the restart: alpha 20/20, fofos 8/8,
|
||||||
|
production-pertento-bridge 15/15, deadpool 9/9, macbookpro 6/6. No process came back `errored` and none
|
||||||
|
failed to restart.
|
||||||
|
|
||||||
|
## The plan said `db:push` must report no changes. It did not, on any host.
|
||||||
|
|
||||||
|
This is a finding, and it is **not** what the plan's stop-condition describes, which is why the rollout
|
||||||
|
continued.
|
||||||
|
|
||||||
|
`bun db:push` printed `Changes applied` on all five hosts. Run again in `--strict --verbose`, it wants the
|
||||||
|
same two statements every time:
|
||||||
|
|
||||||
|
```sql
|
||||||
|
ALTER TABLE "command_hq_sessions" ALTER COLUMN "reference_paths" SET DEFAULT '{}';
|
||||||
|
ALTER TABLE "music_now_playing" DROP CONSTRAINT "pk_music_now_playing";
|
||||||
|
ALTER TABLE "music_now_playing" ADD CONSTRAINT "pk_music_now_playing" PRIMARY KEY("user_id","device");
|
||||||
|
```
|
||||||
|
|
||||||
|
**The same diff appears on `officer-dev`, this release's own source host, whose database has been running
|
||||||
|
this schema for days.** That is the decisive check: the plan's condition is "that host and this tree
|
||||||
|
disagree about something else", and no host disagrees with the tree — the tree disagrees with itself,
|
||||||
|
identically, everywhere. It is the drizzle-kit mis-diff already documented in `src/databases/CLAUDE.md`
|
||||||
|
under "Composite keys", here on a named composite **primary** key rather than a unique constraint, plus a
|
||||||
|
spurious array-default diff.
|
||||||
|
|
||||||
|
Verified by query per host rather than by exit code, as the doctrine requires:
|
||||||
|
|
||||||
|
- `alpha` — `pk_music_now_playing` = `PRIMARY KEY (user_id, device)`; `reference_paths` default
|
||||||
|
`'{}'::text[]`. Both exactly what the schema code says.
|
||||||
|
- `fofos`, `production-pertento-bridge`, `deadpool-rezio-bridge`, `macbookpro` — `reference_paths`
|
||||||
|
default `'{}'::text[]`. `music_now_playing` **does not exist** on these four (`information_schema`
|
||||||
|
count 0): the music plugin is not installed there, so there is nothing for that half of the diff to
|
||||||
|
touch. Not a fault.
|
||||||
|
|
||||||
|
So no host's schema is wrong. What is wrong is that `db:push` cannot serve as the proof step this plan
|
||||||
|
wanted it to be — it reports work on a database that already matches.
|
||||||
|
|
||||||
|
**What the owner should know:** on `alpha`, where the table exists, every deploy drops and re-adds a live
|
||||||
|
table's primary key for no reason. Harmless so far and verified intact afterwards, but it is real DDL on
|
||||||
|
real data, repeated each release. The fix belongs in the schema code, not in a rollout.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- `fofos` carries one untracked file, `scripts/chk.ts`. Noted, not a reason to skip — a fast-forward
|
||||||
|
cannot touch it.
|
||||||
|
- `bun run plugins:update` restarts each plugin's sidecar itself. On `alpha` that is 13 sidecars bouncing
|
||||||
|
before the `officer` restart the plan asked for. Listed above as the plugin count; "restarted" is
|
||||||
|
therefore more than `officer` alone on every host but `macbookpro`.
|
||||||
|
- `officer-dev` is **current in tree** at `c813eef6` but its `officer` process was not restarted, so it is
|
||||||
|
still serving the previous bundle. Restarting it is the owner's deliberate last step.
|
||||||
|
|
||||||
|
## Not verified by this deploy
|
||||||
|
|
||||||
|
The plan's visual check — open a `.m4a` and confirm the audio player shows no transcribe icon, while an
|
||||||
|
image still offers OCR and a video still offers extract-audio. That needs a browser and a hard refresh,
|
||||||
|
and no host was checked that way. The code is deployed; that it looks right is unconfirmed.
|
||||||
@@ -0,0 +1,54 @@
|
|||||||
|
# 2026.08.24.02
|
||||||
|
|
||||||
|
Speech-to-text leaves the platform. It is a plugin now — `transcribe`, in the catalogue.
|
||||||
|
|
||||||
|
**No schema change.** Nothing under `src/databases/**/schema.ts` moved. `bun db:push` stays a step because
|
||||||
|
it is cheap and proves the database agrees with the code, but it must report no changes.
|
||||||
|
|
||||||
|
**No dependency change.** `bun.lock` is untouched. **Do not run `bun install`.**
|
||||||
|
|
||||||
|
## What is in it
|
||||||
|
|
||||||
|
- **`api/stt/` and `server-settings/stt.ts` are gone**, along with `POST /chat/stt`,
|
||||||
|
`POST /file-browser/transcribe`, the FileViewer handler behind the button hidden last release, the
|
||||||
|
Settings → Speech to Text screen and the orphaned locale keys in all six locales.
|
||||||
|
- **The transcription logic moved byte for byte** into `plugins/transcribe` — the two-pass
|
||||||
|
detect-then-transcribe, the 25 language priming prompts, the timeouts. Verified identical, not rewritten.
|
||||||
|
- **`transcribe` is in the catalogue**, so it can be installed from `/plugins` on any host that wants it.
|
||||||
|
|
||||||
|
## Installing the plugin is NOT part of this release
|
||||||
|
|
||||||
|
This release only removes. A host that had speech-to-text configured loses it until somebody installs the
|
||||||
|
plugin and points it at a server — and no host here had it configured, so nothing is lost anywhere.
|
||||||
|
|
||||||
|
**Do not install it as part of the rollout.** Which hosts get transcription is the owner's decision, made
|
||||||
|
per host, and an install grants access to every role. Report that the catalogue entry is available; leave
|
||||||
|
it there.
|
||||||
|
|
||||||
|
## Every host
|
||||||
|
|
||||||
|
1. `git fetch origin --tags` then `git merge --ff-only deploy/2026.08.24.02`
|
||||||
|
2. `bun db:push` — expect **no changes**. If it wants to alter something, stop and report it: this release
|
||||||
|
touched no schema, so a diff means that host and this tree disagree about something else.
|
||||||
|
3. `bun run plugins:update`
|
||||||
|
4. `pm2 restart officer`
|
||||||
|
|
||||||
|
`officer` only at step 4. Step 3 restarts each plugin's own sidecar, which belongs in the report.
|
||||||
|
|
||||||
|
**`macbookpro`:** append `/opt/homebrew/bin` to PATH for the `bun` commands, run `pm2` separately.
|
||||||
|
|
||||||
|
## What to check, this release
|
||||||
|
|
||||||
|
**Settings should no longer offer Speech to Text.** That screen is gone, and its absence is the visible
|
||||||
|
change. If it is still there, the frontend bundle is stale — hard refresh before deciding it failed.
|
||||||
|
|
||||||
|
The audio player's header should still show download and close, and still no transcribe icon. Images
|
||||||
|
should still offer OCR and videos extract audio; those stay in the platform for now.
|
||||||
|
|
||||||
|
No direct SQL is needed and none is authorised.
|
||||||
|
|
||||||
|
## Hosts
|
||||||
|
|
||||||
|
- **`edge-pertento-bridge`** — skipped, per `docs/servers.md`.
|
||||||
|
- **`officer-dev`** — skip it. Restarting `officer` on the host running the rollout ends the rollout. It
|
||||||
|
already has this code and the plugin installed; name it as skipped and say so.
|
||||||
+1
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "officer",
|
"name": "officer",
|
||||||
"version": "2026.08.24.01",
|
"version": "2026.08.24.02",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"workspaces": [
|
"workspaces": [
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import {
|
|||||||
LayoutGrid,
|
LayoutGrid,
|
||||||
Lock,
|
Lock,
|
||||||
Mail,
|
Mail,
|
||||||
Mic,
|
|
||||||
ScanText,
|
ScanText,
|
||||||
ShieldCheck,
|
ShieldCheck,
|
||||||
Sparkles,
|
Sparkles,
|
||||||
@@ -156,7 +155,6 @@ const SETTINGS_ITEMS: SettingsItem[] = [
|
|||||||
},
|
},
|
||||||
// System — SystemSettings.tsx
|
// System — SystemSettings.tsx
|
||||||
{ id: 'set:tts', labelKey: 'settings.sections.tts.label', to: '/settings/system/tts', icon: Volume2 },
|
{ id: 'set:tts', labelKey: 'settings.sections.tts.label', to: '/settings/system/tts', icon: Volume2 },
|
||||||
{ id: 'set:stt', labelKey: 'settings.sections.stt.label', to: '/settings/system/stt', icon: Mic },
|
|
||||||
{ id: 'set:ocr', labelKey: 'settings.sections.ocr.label', to: '/settings/system/ocr', icon: ScanText },
|
{ id: 'set:ocr', labelKey: 'settings.sections.ocr.label', to: '/settings/system/ocr', icon: ScanText },
|
||||||
{
|
{
|
||||||
id: 'set:maintenance',
|
id: 'set:maintenance',
|
||||||
|
|||||||
@@ -1,107 +0,0 @@
|
|||||||
import type { FormEvent } from 'react';
|
|
||||||
import { useState, useEffect } from 'react';
|
|
||||||
import { useQuery, useQueryClient } from '@tanstack/react-query';
|
|
||||||
import { toast } from 'sonner';
|
|
||||||
import { Button } from '@/components/ui/button';
|
|
||||||
import { Input } from '@/components/ui/input';
|
|
||||||
import { Label } from '@/components/ui/label';
|
|
||||||
import { useClient } from 'hooks/useClient';
|
|
||||||
import { useTranslation } from '@/lib/i18n';
|
|
||||||
|
|
||||||
type SttConfig = {
|
|
||||||
url: string;
|
|
||||||
};
|
|
||||||
|
|
||||||
export const STTSection = () => {
|
|
||||||
const { t } = useTranslation();
|
|
||||||
const client = useClient();
|
|
||||||
const queryClient = useQueryClient();
|
|
||||||
|
|
||||||
const { data: config, isLoading } = useQuery({
|
|
||||||
queryKey: ['STT_CONFIG'],
|
|
||||||
queryFn: () => client.get<SttConfig | null>('/server-settings/stt'),
|
|
||||||
});
|
|
||||||
|
|
||||||
const [url, setUrl] = useState('http://localhost:64201');
|
|
||||||
const [isSaving, setIsSaving] = useState(false);
|
|
||||||
const [isTesting, setIsTesting] = useState(false);
|
|
||||||
|
|
||||||
useEffect(() => {
|
|
||||||
if (!config) return;
|
|
||||||
setUrl(config.url);
|
|
||||||
}, [config]);
|
|
||||||
|
|
||||||
const handleSave = async (ev?: FormEvent) => {
|
|
||||||
ev?.preventDefault();
|
|
||||||
if (isSaving) return;
|
|
||||||
setIsSaving(true);
|
|
||||||
try {
|
|
||||||
await client.put('/server-settings/stt', { url });
|
|
||||||
queryClient.invalidateQueries({ queryKey: ['STT_CONFIG'] });
|
|
||||||
toast.success(t('settings.server.stt.saved'));
|
|
||||||
} catch {
|
|
||||||
toast.error(t('settings.server.stt.saveFailed'));
|
|
||||||
} finally {
|
|
||||||
setIsSaving(false);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const handleTest = async () => {
|
|
||||||
if (isTesting) return;
|
|
||||||
setIsTesting(true);
|
|
||||||
try {
|
|
||||||
const res = await client.post<{ success?: boolean; error?: string }>('/server-settings/stt/test', { url });
|
|
||||||
if (res.error) {
|
|
||||||
toast.error(res.error);
|
|
||||||
} else {
|
|
||||||
toast.success(t('settings.server.stt.reachable'));
|
|
||||||
}
|
|
||||||
} catch (err: unknown) {
|
|
||||||
const raw = (err as { message?: string })?.message;
|
|
||||||
let msg = 'Connection failed';
|
|
||||||
try {
|
|
||||||
if (raw) msg = JSON.parse(raw).error ?? msg;
|
|
||||||
} catch {
|
|
||||||
/* ignore */
|
|
||||||
}
|
|
||||||
toast.error(msg);
|
|
||||||
} finally {
|
|
||||||
setIsTesting(false);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
if (isLoading) return <p className="text-sm text-duck-dark/40">{t('settings.server.loading')}</p>;
|
|
||||||
|
|
||||||
return (
|
|
||||||
<form onSubmit={handleSave} className="grid gap-4">
|
|
||||||
<Label className="grid gap-2">
|
|
||||||
<span className="text-duck-dark/70 dark:text-foreground/70">{t('settings.server.stt.serverUrl')}</span>
|
|
||||||
<Input
|
|
||||||
className="h-11 bg-background/60 border-duck-dark/20 text-duck-dark dark:text-foreground placeholder:text-duck-dark/40"
|
|
||||||
placeholder="http://localhost:64201"
|
|
||||||
value={url}
|
|
||||||
onChange={(ev) => setUrl(ev.target.value)}
|
|
||||||
/>
|
|
||||||
</Label>
|
|
||||||
|
|
||||||
<div className="flex items-center gap-2">
|
|
||||||
<Button
|
|
||||||
type="submit"
|
|
||||||
disabled={isSaving}
|
|
||||||
className="flex-1 h-11 bg-duck-yellow hover:bg-duck-yellow/90 text-duck-teal font-bold transition-all duration-200 hover:scale-105 cursor-pointer disabled:opacity-50 disabled:hover:scale-100"
|
|
||||||
>
|
|
||||||
{isSaving ? 'Saving...' : 'Save'}
|
|
||||||
</Button>
|
|
||||||
<Button
|
|
||||||
type="button"
|
|
||||||
variant="outline"
|
|
||||||
onClick={handleTest}
|
|
||||||
disabled={isTesting}
|
|
||||||
className="h-11 cursor-pointer disabled:opacity-50"
|
|
||||||
>
|
|
||||||
{isTesting ? 'Testing...' : 'Test'}
|
|
||||||
</Button>
|
|
||||||
</div>
|
|
||||||
</form>
|
|
||||||
);
|
|
||||||
};
|
|
||||||
@@ -1,5 +1,5 @@
|
|||||||
import { useMemo } from 'react';
|
import { useMemo } from 'react';
|
||||||
import { Bot, Hammer, Mail, Mic, ScanText, Settings, Volume2 } from 'lucide-react';
|
import { Bot, Hammer, Mail, ScanText, Settings, Volume2 } from 'lucide-react';
|
||||||
import type { LayoutNode, PanelComponents } from 'officerdev';
|
import type { LayoutNode, PanelComponents } from 'officerdev';
|
||||||
import { WorkspaceLayout } from 'officerdev';
|
import { WorkspaceLayout } from 'officerdev';
|
||||||
|
|
||||||
@@ -7,7 +7,6 @@ import { SettingsRoute, createSettingsPanelComponents, type SettingsSectionGroup
|
|||||||
import { SMTPSection } from './ServerSettings/SMTPSection';
|
import { SMTPSection } from './ServerSettings/SMTPSection';
|
||||||
import { MaintenanceSection } from './ServerSettings/MaintenanceSection';
|
import { MaintenanceSection } from './ServerSettings/MaintenanceSection';
|
||||||
import { TTSSection } from './ServerSettings/TTSSection';
|
import { TTSSection } from './ServerSettings/TTSSection';
|
||||||
import { STTSection } from './ServerSettings/STTSection';
|
|
||||||
import { OCRSection } from './ServerSettings/OCRSection';
|
import { OCRSection } from './ServerSettings/OCRSection';
|
||||||
|
|
||||||
const groups: SettingsSectionGroup[] = [
|
const groups: SettingsSectionGroup[] = [
|
||||||
@@ -22,13 +21,6 @@ const groups: SettingsSectionGroup[] = [
|
|||||||
descriptionKey: 'settings.sections.tts.description',
|
descriptionKey: 'settings.sections.tts.description',
|
||||||
content: <TTSSection />,
|
content: <TTSSection />,
|
||||||
},
|
},
|
||||||
{
|
|
||||||
key: 'stt',
|
|
||||||
icon: Mic,
|
|
||||||
titleKey: 'settings.sections.stt.label',
|
|
||||||
descriptionKey: 'settings.sections.stt.description',
|
|
||||||
content: <STTSection />,
|
|
||||||
},
|
|
||||||
{
|
{
|
||||||
key: 'ocr',
|
key: 'ocr',
|
||||||
icon: ScanText,
|
icon: ScanText,
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Text-zu-Sprache",
|
"label": "Text-zu-Sprache",
|
||||||
"description": "TTS-Anbieter und Stimme"
|
"description": "TTS-Anbieter und Stimme"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Sprache-zu-Text",
|
|
||||||
"description": "Whisper-Server-URL"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Vision-Modell für Textextraktion"
|
"description": "Vision-Modell für Textextraktion"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "OCR-Einstellungen konnten nicht gespeichert werden",
|
"saveFailed": "OCR-Einstellungen konnten nicht gespeichert werden",
|
||||||
"reachable": "Vision-Modell-Server ist erreichbar"
|
"reachable": "Vision-Modell-Server ist erreichbar"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "Whisper-Server-URL",
|
|
||||||
"saved": "STT-Einstellungen gespeichert",
|
|
||||||
"saveFailed": "STT-Einstellungen konnten nicht gespeichert werden",
|
|
||||||
"reachable": "Whisper-Server ist erreichbar"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "OpenAI-kompatibel (Kokoro, OpenAI usw.)",
|
"openaiCompatible": "OpenAI-kompatibel (Kokoro, OpenAI usw.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Text to Speech",
|
"label": "Text to Speech",
|
||||||
"description": "TTS provider and voice"
|
"description": "TTS provider and voice"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Speech to Text",
|
|
||||||
"description": "Whisper server URL"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Vision model for text extraction"
|
"description": "Vision model for text extraction"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "Failed to save OCR settings",
|
"saveFailed": "Failed to save OCR settings",
|
||||||
"reachable": "Vision model server is reachable"
|
"reachable": "Vision model server is reachable"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "Whisper Server URL",
|
|
||||||
"saved": "STT settings saved",
|
|
||||||
"saveFailed": "Failed to save STT settings",
|
|
||||||
"reachable": "Whisper server is reachable"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "OpenAI-compatible (Kokoro, OpenAI, etc.)",
|
"openaiCompatible": "OpenAI-compatible (Kokoro, OpenAI, etc.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Texto a voz",
|
"label": "Texto a voz",
|
||||||
"description": "Proveedor de TTS y voz"
|
"description": "Proveedor de TTS y voz"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Voz a texto",
|
|
||||||
"description": "URL del servidor Whisper"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Modelo de visión para extracción de texto"
|
"description": "Modelo de visión para extracción de texto"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "No se pudieron guardar los ajustes de OCR",
|
"saveFailed": "No se pudieron guardar los ajustes de OCR",
|
||||||
"reachable": "El servidor del modelo de visión está accesible"
|
"reachable": "El servidor del modelo de visión está accesible"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "URL del servidor Whisper",
|
|
||||||
"saved": "Ajustes de STT guardados",
|
|
||||||
"saveFailed": "No se pudieron guardar los ajustes de STT",
|
|
||||||
"reachable": "El servidor Whisper está accesible"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "Compatible con OpenAI (Kokoro, OpenAI, etc.)",
|
"openaiCompatible": "Compatible con OpenAI (Kokoro, OpenAI, etc.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Synthèse vocale",
|
"label": "Synthèse vocale",
|
||||||
"description": "Fournisseur et voix pour la synthèse vocale"
|
"description": "Fournisseur et voix pour la synthèse vocale"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Reconnaissance vocale",
|
|
||||||
"description": "URL du serveur Whisper"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Modèle de vision pour l'extraction de texte"
|
"description": "Modèle de vision pour l'extraction de texte"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "Échec de l'enregistrement des paramètres OCR",
|
"saveFailed": "Échec de l'enregistrement des paramètres OCR",
|
||||||
"reachable": "Le serveur du modèle de vision est accessible"
|
"reachable": "Le serveur du modèle de vision est accessible"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "URL du serveur Whisper",
|
|
||||||
"saved": "Paramètres de reconnaissance vocale enregistrés",
|
|
||||||
"saveFailed": "Échec de l'enregistrement des paramètres de reconnaissance vocale",
|
|
||||||
"reachable": "Le serveur Whisper est accessible"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "Compatible OpenAI (Kokoro, OpenAI, etc.)",
|
"openaiCompatible": "Compatible OpenAI (Kokoro, OpenAI, etc.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Texto para fala",
|
"label": "Texto para fala",
|
||||||
"description": "Fornecedor de TTS e voz"
|
"description": "Fornecedor de TTS e voz"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Fala para texto",
|
|
||||||
"description": "URL do servidor Whisper"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Modelo de visão para extração de texto"
|
"description": "Modelo de visão para extração de texto"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "Falha ao guardar as definições de OCR",
|
"saveFailed": "Falha ao guardar as definições de OCR",
|
||||||
"reachable": "O servidor do modelo de visão está acessível"
|
"reachable": "O servidor do modelo de visão está acessível"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "URL do servidor Whisper",
|
|
||||||
"saved": "Definições de STT guardadas",
|
|
||||||
"saveFailed": "Falha ao guardar as definições de STT",
|
|
||||||
"reachable": "O servidor Whisper está acessível"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "Compatível com OpenAI (Kokoro, OpenAI, etc.)",
|
"openaiCompatible": "Compatível com OpenAI (Kokoro, OpenAI, etc.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -145,10 +145,6 @@
|
|||||||
"label": "Text till tal",
|
"label": "Text till tal",
|
||||||
"description": "TTS-leverantör och röst"
|
"description": "TTS-leverantör och röst"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"label": "Tal till text",
|
|
||||||
"description": "Whisper-serverns URL"
|
|
||||||
},
|
|
||||||
"ocr": {
|
"ocr": {
|
||||||
"label": "OCR",
|
"label": "OCR",
|
||||||
"description": "Synmodell för textextrahering"
|
"description": "Synmodell för textextrahering"
|
||||||
@@ -234,12 +230,6 @@
|
|||||||
"saveFailed": "Det gick inte att spara OCR-inställningarna",
|
"saveFailed": "Det gick inte att spara OCR-inställningarna",
|
||||||
"reachable": "Synmodellsservern är nåbar"
|
"reachable": "Synmodellsservern är nåbar"
|
||||||
},
|
},
|
||||||
"stt": {
|
|
||||||
"serverUrl": "URL till Whisper-server",
|
|
||||||
"saved": "STT-inställningarna sparade",
|
|
||||||
"saveFailed": "Det gick inte att spara STT-inställningarna",
|
|
||||||
"reachable": "Whisper-servern är nåbar"
|
|
||||||
},
|
|
||||||
"tts": {
|
"tts": {
|
||||||
"openaiCompatible": "OpenAI-kompatibel (Kokoro, OpenAI, m.fl.)",
|
"openaiCompatible": "OpenAI-kompatibel (Kokoro, OpenAI, m.fl.)",
|
||||||
"elevenLabs": "ElevenLabs",
|
"elevenLabs": "ElevenLabs",
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ import { createRouter } from '../../create-router';
|
|||||||
import * as errors from '@@/custom-errors';
|
import * as errors from '@@/custom-errors';
|
||||||
import * as sidecar from '@@/sidecar-registry';
|
import * as sidecar from '@@/sidecar-registry';
|
||||||
import {
|
import {
|
||||||
getUserSettings,
|
|
||||||
getChatSessionModel,
|
getChatSessionModel,
|
||||||
getChatSessionModels,
|
getChatSessionModels,
|
||||||
listCommandHqSessionIds,
|
listCommandHqSessionIds,
|
||||||
@@ -45,7 +44,6 @@ import type { ChatIdentity } from './claude-sessions';
|
|||||||
import { officerModelForApiModel } from './model-aliases';
|
import { officerModelForApiModel } from './model-aliases';
|
||||||
import { readAccountEmail } from './claude-usage';
|
import { readAccountEmail } from './claude-usage';
|
||||||
import { listStoredUsage, mergeUsage, snapshotClaudeUsage } from './usage-store';
|
import { listStoredUsage, mergeUsage, snapshotClaudeUsage } from './usage-store';
|
||||||
import { readSttConfig } from '../server-settings/stt';
|
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Whose transcripts a request may read.
|
* Whose transcripts a request may read.
|
||||||
@@ -81,7 +79,6 @@ export async function chatIdentity(user: { id: number; email: string }): Promise
|
|||||||
// identity the way `spawnClaudeAsMember` does. Restrict here rather than at the permission layer because
|
// identity the way `spawnClaudeAsMember` does. Restrict here rather than at the permission layer because
|
||||||
// `chat` is one permission covering both harnesses, and splitting it would strand the grants already issued.
|
// `chat` is one permission covering both harnesses, and splitting it would strand the grants already issued.
|
||||||
// The matching refusal on the execution path is in `websocket.ts` → `handleChat`.
|
// The matching refusal on the execution path is in `websocket.ts` → `handleChat`.
|
||||||
import { transcribeAudio } from '../stt/transcribe';
|
|
||||||
import { registerAgentPanelRoutes } from './agent-panels-routes';
|
import { registerAgentPanelRoutes } from './agent-panels-routes';
|
||||||
|
|
||||||
export const chatRouter = createRouter();
|
export const chatRouter = createRouter();
|
||||||
@@ -629,32 +626,6 @@ chatRouter.get('/models', async (ctx: Context) => {
|
|||||||
}
|
}
|
||||||
});
|
});
|
||||||
|
|
||||||
// POST /chat/stt — proxy an audio clip to the configured Whisper server.
|
|
||||||
chatRouter.post('/stt', async (ctx: Context) => {
|
|
||||||
const sttConfig = await readSttConfig();
|
|
||||||
if (!sttConfig?.url) {
|
|
||||||
return ctx.json({ error: 'Whisper not configured — set it up in Settings → Speech to Text' }, 400);
|
|
||||||
}
|
|
||||||
|
|
||||||
const user = ctx.get('user');
|
|
||||||
const body = await ctx.req.parseBody();
|
|
||||||
const file = body['file'];
|
|
||||||
if (!file || !(file instanceof File)) {
|
|
||||||
return ctx.json({ error: 'file is required' }, 400);
|
|
||||||
}
|
|
||||||
|
|
||||||
const settings = (await getUserSettings(user.id)) as { languages?: { spoken?: string[] } };
|
|
||||||
const spokenLanguages = settings.languages?.spoken ?? [];
|
|
||||||
|
|
||||||
try {
|
|
||||||
const result = await transcribeAudio({ file, whisperUrl: sttConfig.url, spokenLanguages });
|
|
||||||
return ctx.json(result);
|
|
||||||
} catch (err) {
|
|
||||||
logger.error('STT proxy failed', { error: String(err) });
|
|
||||||
return ctx.json({ error: err instanceof Error ? err.message : 'Failed to reach Whisper server' }, 502);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
|
|
||||||
// The agent address book — naming a chat panel so other agents on the same dashboard can reach it.
|
// The agent address book — naming a chat panel so other agents on the same dashboard can reach it.
|
||||||
// Lives on this router because it is the same authority `chat` already grants: creating and naming
|
// Lives on this router because it is the same authority `chat` already grants: creating and naming
|
||||||
// Claude sessions. See servers/api/chat/agent-panels-routes.ts and docs/agent-coordination.md.
|
// Claude sessions. See servers/api/chat/agent-panels-routes.ts and docs/agent-coordination.md.
|
||||||
|
|||||||
@@ -6,9 +6,7 @@ import { getOwnerHomeDir, DATA_PATH } from '@@/data-path';
|
|||||||
import { resolveHomeDir } from '@@/user-home';
|
import { resolveHomeDir } from '@@/user-home';
|
||||||
import * as errors from '@@/custom-errors';
|
import * as errors from '@@/custom-errors';
|
||||||
import { readTtsConfig } from '@@/api/server-settings/tts';
|
import { readTtsConfig } from '@@/api/server-settings/tts';
|
||||||
import { readSttConfig } from '@@/api/server-settings/stt';
|
|
||||||
import { readOcrConfig } from '@@/api/server-settings/ocr';
|
import { readOcrConfig } from '@@/api/server-settings/ocr';
|
||||||
import { transcribeAudio } from '@@/api/stt/transcribe';
|
|
||||||
import { getUserSettings } from 'officerdb';
|
import { getUserSettings } from 'officerdb';
|
||||||
|
|
||||||
async function getUserTtsVoice(userId: number): Promise<string | null> {
|
async function getUserTtsVoice(userId: number): Promise<string | null> {
|
||||||
@@ -1104,46 +1102,6 @@ router.post('/extract', async (ctx) => {
|
|||||||
return ctx.json({ extractedPath });
|
return ctx.json({ extractedPath });
|
||||||
});
|
});
|
||||||
|
|
||||||
// Transcribe audio via Whisper with caching
|
|
||||||
// Workflow: detect language → check user's spoken languages → translate if needed → transcribe
|
|
||||||
router.post('/transcribe', async (ctx) => {
|
|
||||||
const user = ctx.get('user');
|
|
||||||
const { path: filePath, root, saveNextTo } = ctx.get('body') as { path: string; root?: string; saveNextTo?: boolean };
|
|
||||||
if (!filePath) throw errors.BAD_REQUEST('path is required');
|
|
||||||
|
|
||||||
const rootDir = getRootDir(user, root);
|
|
||||||
const absPath = resolveUserPath(rootDir, filePath);
|
|
||||||
const s = await stat(absPath);
|
|
||||||
if (s.isDirectory()) throw errors.BAD_REQUEST('Cannot transcribe a directory');
|
|
||||||
|
|
||||||
// No caching: a transcription is always produced fresh and overwrites any previous one.
|
|
||||||
const { dir, name } = parsePath(filePath.replace(/^\/+/, ''));
|
|
||||||
|
|
||||||
const sttConfig = await readSttConfig();
|
|
||||||
const whisperUrl = sttConfig?.url;
|
|
||||||
if (!whisperUrl) throw errors.BAD_REQUEST('Whisper not configured — set it up in Settings → Speech to Text');
|
|
||||||
|
|
||||||
const settings = await getUserSettings(user.id);
|
|
||||||
const spokenLanguages = (settings.languages as { spoken?: string[] })?.spoken ?? [];
|
|
||||||
|
|
||||||
let text: string;
|
|
||||||
try {
|
|
||||||
const result = await transcribeAudio({ file: Bun.file(absPath), whisperUrl, spokenLanguages });
|
|
||||||
text = result.text;
|
|
||||||
} catch (err) {
|
|
||||||
throw errors.BAD_REQUEST(err instanceof Error ? err.message : 'Transcription failed');
|
|
||||||
}
|
|
||||||
|
|
||||||
if (saveNextTo) {
|
|
||||||
const siblingRel = dir ? `${dir}/${name}.md` : `${name}.md`;
|
|
||||||
const siblingAbs = resolveUserPath(rootDir, siblingRel);
|
|
||||||
await Bun.write(siblingAbs, text);
|
|
||||||
return ctx.json({ transcriptionPath: `/${siblingRel}`, transcriptionRoot: root ?? 'home', cached: false });
|
|
||||||
}
|
|
||||||
|
|
||||||
return ctx.json({ text, cached: false });
|
|
||||||
});
|
|
||||||
|
|
||||||
// Search files by name
|
// Search files by name
|
||||||
router.get('/search', async (ctx) => {
|
router.get('/search', async (ctx) => {
|
||||||
const user = ctx.get('user');
|
const user = ctx.get('user');
|
||||||
|
|||||||
@@ -9,7 +9,6 @@ import { readServerSettings, writeServerSettings } from 'officerdb';
|
|||||||
import { chatProvidersRouter } from './chat-providers';
|
import { chatProvidersRouter } from './chat-providers';
|
||||||
import { smtpRouter } from './smtp';
|
import { smtpRouter } from './smtp';
|
||||||
import { ttsRouter } from './tts';
|
import { ttsRouter } from './tts';
|
||||||
import { sttRouter } from './stt';
|
|
||||||
import { ocrRouter } from './ocr';
|
import { ocrRouter } from './ocr';
|
||||||
|
|
||||||
export const serverSettingsRouter = createRouter();
|
export const serverSettingsRouter = createRouter();
|
||||||
@@ -17,7 +16,6 @@ export const serverSettingsRouter = createRouter();
|
|||||||
serverSettingsRouter.route('/chat-providers', chatProvidersRouter);
|
serverSettingsRouter.route('/chat-providers', chatProvidersRouter);
|
||||||
serverSettingsRouter.route('/smtp', smtpRouter);
|
serverSettingsRouter.route('/smtp', smtpRouter);
|
||||||
serverSettingsRouter.route('/tts', ttsRouter);
|
serverSettingsRouter.route('/tts', ttsRouter);
|
||||||
serverSettingsRouter.route('/stt', sttRouter);
|
|
||||||
serverSettingsRouter.route('/ocr', ocrRouter);
|
serverSettingsRouter.route('/ocr', ocrRouter);
|
||||||
|
|
||||||
export { readServerSettings as readSettings };
|
export { readServerSettings as readSettings };
|
||||||
|
|||||||
@@ -1,48 +0,0 @@
|
|||||||
import { createRouter } from '../../create-router';
|
|
||||||
import { readServerSettings, writeServerSettings } from 'officerdb';
|
|
||||||
|
|
||||||
type SttConfig = {
|
|
||||||
url: string;
|
|
||||||
};
|
|
||||||
|
|
||||||
export async function readSttConfig(): Promise<SttConfig | undefined> {
|
|
||||||
const settings = await readServerSettings();
|
|
||||||
return settings.stt as SttConfig | undefined;
|
|
||||||
}
|
|
||||||
|
|
||||||
export const sttRouter = createRouter();
|
|
||||||
|
|
||||||
sttRouter.get('/', async (ctx) => {
|
|
||||||
const stt = await readSttConfig();
|
|
||||||
if (!stt) return ctx.json(null);
|
|
||||||
return ctx.json(stt);
|
|
||||||
});
|
|
||||||
|
|
||||||
sttRouter.put('/', async (ctx) => {
|
|
||||||
const body = await ctx.req.json<SttConfig>();
|
|
||||||
const settings = await readServerSettings();
|
|
||||||
settings.stt = body;
|
|
||||||
await writeServerSettings(settings);
|
|
||||||
return ctx.json({ success: true });
|
|
||||||
});
|
|
||||||
|
|
||||||
sttRouter.post('/test', async (ctx) => {
|
|
||||||
const body = await ctx.req.json<{ url: string }>();
|
|
||||||
if (!body.url) return ctx.json({ error: 'URL required' }, 400);
|
|
||||||
|
|
||||||
try {
|
|
||||||
const controller = new AbortController();
|
|
||||||
const timeout = setTimeout(() => controller.abort(), 5000);
|
|
||||||
const res = await fetch(`${body.url.replace(/\/+$/, '')}/inference`, {
|
|
||||||
method: 'POST',
|
|
||||||
body: new FormData(),
|
|
||||||
signal: controller.signal,
|
|
||||||
});
|
|
||||||
clearTimeout(timeout);
|
|
||||||
// Whisper will return an error for empty form, but a response means it's reachable
|
|
||||||
return ctx.json({ success: true, status: res.status });
|
|
||||||
} catch (err) {
|
|
||||||
const message = err instanceof Error ? err.message : 'Unknown error';
|
|
||||||
return ctx.json({ error: message }, 500);
|
|
||||||
}
|
|
||||||
});
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
import { describe, expect, it } from 'bun:test';
|
|
||||||
import { toParagraphs } from './transcribe';
|
|
||||||
import type { WhisperSegment } from './transcribe';
|
|
||||||
|
|
||||||
// Whisper's own shape: leading space on every segment, times in seconds.
|
|
||||||
const seg = (text: string, start: number, end: number, extra: Partial<WhisperSegment> = {}): WhisperSegment => ({
|
|
||||||
text,
|
|
||||||
start,
|
|
||||||
end,
|
|
||||||
...extra,
|
|
||||||
});
|
|
||||||
|
|
||||||
describe('toParagraphs', () => {
|
|
||||||
it('joins segments spoken without a real pause into one paragraph', () => {
|
|
||||||
const out = toParagraphs([seg(' I went to the shop.', 0, 2), seg(' It was closed.', 2.2, 4)]);
|
|
||||||
expect(out).toBe('I went to the shop. It was closed.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('starts a new paragraph after a long silence', () => {
|
|
||||||
const out = toParagraphs([seg(' First thought.', 0, 2), seg(' Second thought.', 5, 7)]);
|
|
||||||
expect(out).toBe('First thought.\n\nSecond thought.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('does not break on a breath-length pause', () => {
|
|
||||||
const out = toParagraphs([seg(' Hang on,', 0, 1), seg(' let me think.', 1.5, 3)]);
|
|
||||||
expect(out).toBe('Hang on, let me think.');
|
|
||||||
});
|
|
||||||
|
|
||||||
// The lone "." lines in the old response_format=text output came from segments like this.
|
|
||||||
it('attaches stranded punctuation without a space in front of it', () => {
|
|
||||||
const out = toParagraphs([seg(' See you tomorrow morning', 0, 2), seg(' .', 2.1, 2.2)]);
|
|
||||||
expect(out).toBe('See you tomorrow morning.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('drops segments that are probably silence decoded with low confidence', () => {
|
|
||||||
const out = toParagraphs([
|
|
||||||
seg(' Real speech here.', 0, 2),
|
|
||||||
seg(' Thanks for watching!', 2.1, 3, { no_speech_prob: 0.95, avg_logprob: -1.8 }),
|
|
||||||
seg(' More real speech.', 3.1, 5),
|
|
||||||
]);
|
|
||||||
expect(out).toBe('Real speech here. More real speech.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('keeps a confident segment even when it looks like silence', () => {
|
|
||||||
const out = toParagraphs([seg(' Definitely said.', 0, 2, { no_speech_prob: 0.95, avg_logprob: -0.2 })]);
|
|
||||||
expect(out).toBe('Definitely said.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('keeps segments that report neither probability', () => {
|
|
||||||
const out = toParagraphs([seg(' No metadata at all.', 0, 2)]);
|
|
||||||
expect(out).toBe('No metadata at all.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('measures the pause from the last kept segment, not the dropped one', () => {
|
|
||||||
const out = toParagraphs([
|
|
||||||
seg(' Before.', 0, 2),
|
|
||||||
seg(' Hallucinated.', 2.1, 9, { no_speech_prob: 0.9, avg_logprob: -2 }),
|
|
||||||
seg(' After.', 9.1, 11),
|
|
||||||
]);
|
|
||||||
// The gap that matters is 2 -> 9.1, so these are separate thoughts.
|
|
||||||
expect(out).toBe('Before.\n\nAfter.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('collapses internal whitespace and ignores empty segments', () => {
|
|
||||||
const out = toParagraphs([seg(' Too many\n spaces. ', 0, 2), seg(' ', 2.1, 2.2)]);
|
|
||||||
expect(out).toBe('Too many spaces.');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('returns an empty string when every segment is filtered out', () => {
|
|
||||||
expect(toParagraphs([])).toBe('');
|
|
||||||
expect(toParagraphs([seg(' ', 0, 1)])).toBe('');
|
|
||||||
});
|
|
||||||
|
|
||||||
it('falls back to no break when segments carry no timings', () => {
|
|
||||||
const out = toParagraphs([{ text: ' One.' }, { text: ' Two.' }]);
|
|
||||||
expect(out).toBe('One. Two.');
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,416 +0,0 @@
|
|||||||
// Whisper transcription, always in the audio's own language.
|
|
||||||
//
|
|
||||||
// Detect the language first, then transcribe in it. Whisper's translate mode is English-only, so it
|
|
||||||
// cannot honour "translate into <language>" for anything else — rather than silently returning a
|
|
||||||
// rough English rendering of Portuguese audio, transcription is always faithful to the source.
|
|
||||||
//
|
|
||||||
// The detection must win when Whisper is confident. `spokenLanguages` only breaks ties on clips
|
|
||||||
// Whisper is genuinely unsure about, where its low-confidence guess defaults to English.
|
|
||||||
//
|
|
||||||
// The transcription pass is tuned for one person talking into a phone, which is what this is mostly
|
|
||||||
// used for. See the constants below: the defaults whisper.cpp ships are aimed at clean dictation and
|
|
||||||
// produce loops, clipped words and breath-length paragraph breaks on real voice memos.
|
|
||||||
//
|
|
||||||
// Both requests pass `timeout: false`. Bun's fetch aborts at 300s by default, and whisper.cpp runs at
|
|
||||||
// roughly 8x realtime, so anything over ~40 minutes of audio died with "The operation timed out"
|
|
||||||
// about 20 seconds short of the answer. AbortSignal.timeout() does not raise that ceiling — only
|
|
||||||
// disabling the built-in timeout does.
|
|
||||||
|
|
||||||
// Bun's fetch accepts `timeout: false` to disable its built-in 300s ceiling. The DOM RequestInit
|
|
||||||
// type this project compiles against does not declare it, so widen it here rather than casting at
|
|
||||||
// each call site.
|
|
||||||
type BunRequestInit = RequestInit & { timeout?: boolean | number };
|
|
||||||
|
|
||||||
type TranscribeArgs = {
|
|
||||||
file: File | Blob;
|
|
||||||
whisperUrl: string;
|
|
||||||
spokenLanguages: string[];
|
|
||||||
};
|
|
||||||
|
|
||||||
export type TranscribeResult = {
|
|
||||||
text: string;
|
|
||||||
detectedLanguage: string;
|
|
||||||
};
|
|
||||||
|
|
||||||
// Above this probability Whisper's own detection is taken as authoritative. Below it the clip is
|
|
||||||
// treated as ambiguous and the user's spoken languages break the tie.
|
|
||||||
const CONFIDENT_DETECTION = 0.5;
|
|
||||||
|
|
||||||
// whisper.cpp's "language" field uses full names ("english"); ISO codes live in
|
|
||||||
// `language_probabilities`. Map covers what the user can pick in Profile settings.
|
|
||||||
const LANG_NAME_TO_ISO: Record<string, string> = {
|
|
||||||
english: 'en',
|
|
||||||
portuguese: 'pt',
|
|
||||||
spanish: 'es',
|
|
||||||
french: 'fr',
|
|
||||||
german: 'de',
|
|
||||||
italian: 'it',
|
|
||||||
dutch: 'nl',
|
|
||||||
russian: 'ru',
|
|
||||||
chinese: 'zh',
|
|
||||||
japanese: 'ja',
|
|
||||||
korean: 'ko',
|
|
||||||
};
|
|
||||||
|
|
||||||
// Whisper reuses its own decoded output as the prompt for the next 30-second window, which is what
|
|
||||||
// makes it fall into loops ("I know. I know. I know.") once it slips. `max_context` is the budget for
|
|
||||||
// that carried text, and setting it to 0 stops the loops — but the same switch also disables the
|
|
||||||
// initial prompt, since whisper.cpp gates both on `n_max_text_ctx > 0`.
|
|
||||||
//
|
|
||||||
// The way to keep the prompt and still starve the loop: `carry_initial_prompt` puts the prompt in a
|
|
||||||
// static slot that is filled first and in full, and only the leftover budget goes to decoded text.
|
|
||||||
// Every prompt below is comfortably longer than this budget, so there is never any leftover — the
|
|
||||||
// vocabulary and style reach every window and none of the model's own output does.
|
|
||||||
const PROMPT_CONTEXT_TOKENS = 96;
|
|
||||||
|
|
||||||
// Whisper imitates whatever it is primed with; it does not take instructions. So each prompt is a
|
|
||||||
// worked example of the output wanted — one person talking, punctuated the way it should come back —
|
|
||||||
// and nothing else. An earlier version opened by *describing* the format ("Commas separate clauses,
|
|
||||||
// and apostrophes are written properly") and that backfired: priming words which never occur in
|
|
||||||
// speech put them in play, and a character surname came back from a Monk episode as "Commas".
|
|
||||||
// Everything here is ordinary conversational language for that reason.
|
|
||||||
//
|
|
||||||
// Each is written in its own language, since an English prompt on Portuguese audio biases the
|
|
||||||
// output, and each deliberately runs past PROMPT_CONTEXT_TOKENS — see above for why the length is
|
|
||||||
// load-bearing. Overflow is trimmed from the front, so nothing depends on the opening words.
|
|
||||||
//
|
|
||||||
// Languages without an entry fall through to no prompt at all, which is handled explicitly below.
|
|
||||||
//
|
|
||||||
// Every prompt uses its language's full diacritic set (and, for a non-Latin script, its own
|
|
||||||
// punctuation — Japanese 。/、, Greek ; as a question mark, Arabic ، and ؟) in ordinary words, never
|
|
||||||
// forced in. Omitting one, e.g. writing Swedish without å ä ö, teaches Whisper to drop it too.
|
|
||||||
const TRANSCRIPT_PROMPTS: Record<string, string> = {
|
|
||||||
en:
|
|
||||||
'Hey, sorry for the late reply — I was stuck in a meeting all morning and only just got out. ' +
|
|
||||||
"I had a look at the document you sent over, and honestly, I think it's mostly fine. There are " +
|
|
||||||
"a couple of things I'd change in the second half, but nothing serious. The introduction reads " +
|
|
||||||
'really well. Let me know when you have a few minutes and we can talk it through properly. ' +
|
|
||||||
"I'm around all afternoon, so whenever suits you is fine by me. Thanks again for putting it " +
|
|
||||||
'together, I know it took a while.',
|
|
||||||
pt:
|
|
||||||
'Olá, desculpa a demora a responder — estive numa reunião toda a manhã e só agora consegui ' +
|
|
||||||
'sair. Já vi o documento que enviaste e, sinceramente, acho que está quase tudo bem. Há duas ' +
|
|
||||||
'ou três coisas que mudava na segunda metade, mas nada de grave. A introdução está muito bem ' +
|
|
||||||
'escrita. Diz-me quando tiveres uns minutos e falamos com calma. Estou por aqui toda a tarde, ' +
|
|
||||||
'por isso quando te der jeito está bem para mim. Obrigado outra vez por teres tratado disto, ' +
|
|
||||||
'sei que deu trabalho.',
|
|
||||||
es:
|
|
||||||
'Hola, perdona la tardanza en responder — he estado toda la mañana en una reunión y recién ahora ' +
|
|
||||||
'he podido salir. Ya vi el documento que me enviaste y, la verdad, creo que está bastante bien. ' +
|
|
||||||
'Hay un par de cosas que cambiaría en la segunda mitad, pero nada grave. La introducción está muy ' +
|
|
||||||
'bien escrita. Avísame cuando tengas unos minutos y lo hablamos con calma. Estoy disponible toda ' +
|
|
||||||
'la tarde, así que cuando te venga bien está perfecto para mí. Gracias otra vez por armarlo, sé ' +
|
|
||||||
'que llevó tiempo.',
|
|
||||||
de:
|
|
||||||
'Hallo, entschuldige die späte Antwort — ich saß den ganzen Vormittag in einer Besprechung und ' +
|
|
||||||
'bin gerade erst rausgekommen. Ich habe mir das Dokument angeschaut, das du geschickt hast, und ' +
|
|
||||||
'ehrlich gesagt finde ich es größtenteils gut. In der zweiten Hälfte würde ich ein paar ' +
|
|
||||||
'Kleinigkeiten ändern, aber nichts Ernstes. Die Einleitung liest sich wirklich gut. Sag mir ' +
|
|
||||||
'Bescheid, wenn du später ein paar Minuten hast, dann können wir in Ruhe darüber reden. Ich bin ' +
|
|
||||||
'den ganzen Nachmittag über da, also wann immer es dir passt, ist mir recht. Danke nochmal, dass ' +
|
|
||||||
'du dir die Mühe gemacht hast, ich weiß, das hat Zeit gekostet.',
|
|
||||||
fr:
|
|
||||||
"Salut, désolé pour la réponse tardive — j'étais coincé en réunion toute la matinée et je viens " +
|
|
||||||
"tout juste d'en sortir. J'ai regardé le document que tu as envoyé, et honnêtement, je trouve que " +
|
|
||||||
"c'est plutôt bien. Il y a deux ou trois choses que je changerais dans la deuxième moitié, mais " +
|
|
||||||
"rien de grave. L'introduction est vraiment bien écrite. Dis-moi quand tu as quelques minutes et " +
|
|
||||||
"on en parle tranquillement. Je suis disponible tout l'après-midi, donc quand ça t'arrange, ça me " +
|
|
||||||
"va. Merci encore de l'avoir préparé, je sais que ça a pris du temps.",
|
|
||||||
sv:
|
|
||||||
'Hej, förlåt att jag inte svarat tidigare — jag satt i möte hela förmiddagen och kom precis loss. ' +
|
|
||||||
'Jag har tittat på dokumentet du skickade, och ärligt talat tycker jag att det mesta är bra. Det ' +
|
|
||||||
'är några saker jag skulle ändra i andra halvan, men inget allvarligt. Inledningen är verkligen ' +
|
|
||||||
'bra skriven. Säg till när du har några minuter så pratar vi igenom det ordentligt. Jag är ledig ' +
|
|
||||||
'hela eftermiddagen, så när det passar dig passar det mig. Tack igen för att du satte ihop det, ' +
|
|
||||||
'jag vet att det tog ett tag.',
|
|
||||||
it:
|
|
||||||
'Ciao, scusa il ritardo nella risposta — sono stato in riunione tutta la mattina e sono appena ' +
|
|
||||||
"uscito. Ho dato un'occhiata al documento che mi hai mandato e, sinceramente, penso che vada bene " +
|
|
||||||
'quasi tutto. Ci sono un paio di cose che cambierei nella seconda metà, ma niente di grave. ' +
|
|
||||||
"L'introduzione è scritta davvero bene. Fammi sapere quando hai qualche minuto così ne parliamo " +
|
|
||||||
'con calma. Sono libero tutto il pomeriggio, quindi va bene quando ti fa comodo. Grazie ancora ' +
|
|
||||||
'per averlo messo insieme, so che ci è voluto tempo.',
|
|
||||||
nl:
|
|
||||||
'Hoi, sorry voor het late antwoord — ik zat de hele ochtend in een vergadering en ben er net pas ' +
|
|
||||||
'uit. Ik heb naar het document gekeken dat je hebt gestuurd, en eerlijk gezegd vind ik het ' +
|
|
||||||
'grotendeels goed. Er zijn een paar dingen die ik in de tweede helft zou veranderen, maar niets ' +
|
|
||||||
'ergs. De inleiding leest heel prettig. Laat het me weten wanneer je een paar minuten hebt, dan ' +
|
|
||||||
'bespreken we het rustig. Ik ben de hele middag beschikbaar, dus wanneer het jou uitkomt is voor ' +
|
|
||||||
'mij prima. Bedankt nogmaals dat je het hebt samengesteld, ik weet dat het tijd heeft gekost.',
|
|
||||||
pl:
|
|
||||||
'Cześć, przepraszam za późną odpowiedź — siedziałem cały ranek na spotkaniu i w końcu udało mi ' +
|
|
||||||
'się z niego wyrwać. Zerknąłem na dokument, który przesłałeś, i szczerze mówiąc, myślę, że ' +
|
|
||||||
'większość jest w porządku. Jest kilka rzeczy, które zmieniłbym w drugiej połowie, ale nic ' +
|
|
||||||
'poważnego. Wstęp jest naprawdę dobrze napisany. Daj znać, kiedy będziesz mieć chwilę, możemy to ' +
|
|
||||||
'spokojnie omówić. Jestem wolny całe popołudnie, więc kiedy ci pasuje, mi też pasuje. Dzięki ' +
|
|
||||||
'jeszcze raz za to, że to zebrałeś, wiem, że zajęło to trochę czasu.',
|
|
||||||
da:
|
|
||||||
'Hej, undskyld det sene svar — jeg sad i møde hele formiddagen og er lige kommet ud. Jeg har ' +
|
|
||||||
'kigget på dokumentet, du sendte, og ærligt talt synes jeg det meste er fint. Der er et par ting, ' +
|
|
||||||
'jeg ville ændre i anden halvdel, men ikke noget alvorligt. Indledningen er skrevet rigtig godt. ' +
|
|
||||||
'Sig til, når du har et par minutter, så snakker vi ordentligt om det. Jeg er ledig hele ' +
|
|
||||||
'eftermiddagen, så det passer mig, når det passer dig. Tak igen fordi du satte det sammen, jeg ' +
|
|
||||||
'ved det tog tid.',
|
|
||||||
no:
|
|
||||||
'Hei, beklager det sene svaret — jeg satt i møte hele formiddagen og er akkurat ferdig. Jeg har ' +
|
|
||||||
'sett på dokumentet du sendte, og ærlig talt synes jeg det meste er bra. Det er et par ting jeg ' +
|
|
||||||
'ville endret i andre halvdel, men ingenting alvorlig. Innledningen er skrevet veldig bra. Si ' +
|
|
||||||
'ifra når du har noen minutter, så tar vi en skikkelig prat om det. Jeg er ledig hele ' +
|
|
||||||
'ettermiddagen, så det som passer deg passer meg. Takk igjen for at du satte det sammen, jeg vet ' +
|
|
||||||
'det tok tid.',
|
|
||||||
fi:
|
|
||||||
'Hei, anteeksi myöhäinen vastaus — istuin koko aamupäivän palaverissa ja pääsin juuri äsken pois. ' +
|
|
||||||
'Katsoin dokumentin, jonka lähetit, ja rehellisesti sanottuna suurin osa siitä on ihan hyvä. ' +
|
|
||||||
'Loppupuoliskolla on pari asiaa, jotka muuttaisin, mutta ei mitään vakavaa. Johdanto on ' +
|
|
||||||
'kirjoitettu tosi hyvin. Sano, kun sinulla on hetki aikaa, niin jutellaan siitä kunnolla. Olen ' +
|
|
||||||
'vapaana koko iltapäivän, joten mikä tahansa sopii sinulle, sopii minulle. Kiitos vielä kerran, ' +
|
|
||||||
'kun kokosit sen, tiedän että siihen meni aikaa.',
|
|
||||||
tr:
|
|
||||||
'Selam, geç cevap verdiğim için kusura bakma — bütün sabah toplantıdaydım ve az önce çıkabildim. ' +
|
|
||||||
'Gönderdiğin dokümana baktım ve açıkçası çoğunlukla iyi olduğunu düşünüyorum. İkinci yarıda ' +
|
|
||||||
'değiştireceğim birkaç şey var ama ciddi bir şey değil. Giriş kısmı gerçekten iyi yazılmış. ' +
|
|
||||||
'Birkaç dakikan olduğunda haber ver, düzgünce konuşuruz. Bütün öğleden sonra müsaitim, yani sana ' +
|
|
||||||
'uyan zaman bana da uyar. Bunu hazırladığın için tekrar teşekkürler, biliyorum zaman almıştır.',
|
|
||||||
cs:
|
|
||||||
'Ahoj, promiň za pozdní odpověď — celé dopoledne jsem byl na poradě a teprve teď jsem se dostal ' +
|
|
||||||
'ven. Podíval jsem se na dokument, cos poslal, a upřímně řečeno, myslím, že většina je v pořádku. ' +
|
|
||||||
'V druhé polovině bych pár věcí změnil, ale nic vážného. Úvod je napsaný fakt dobře. Dej vědět, ' +
|
|
||||||
'až budeš mít pár minut, můžeme to pořádně probrat. Celé odpoledne mám volno, takže kdy se ti to ' +
|
|
||||||
'bude hodit, mně to vyhovuje. Díky ještě jednou, žes to dal dohromady, vím, že to zabralo čas.',
|
|
||||||
el:
|
|
||||||
'Γεια σου, συγγνώμη για την καθυστερημένη απάντηση — ήμουν σε συνάντηση όλο το πρωί και μόλις ' +
|
|
||||||
'τώρα βγήκα. Είδα το έγγραφο που έστειλες και, ειλικρινά, νομίζω πως είναι σχεδόν όλο εντάξει. ' +
|
|
||||||
'Υπάρχουν δύο τρία πράγματα που θα άλλαζα στο δεύτερο μισό, αλλά τίποτα σοβαρό. Η εισαγωγή είναι ' +
|
|
||||||
'γραμμένη πολύ καλά. Έχεις λίγα λεπτά να τα πούμε με την ησυχία μας; Είμαι ελεύθερος όλο το ' +
|
|
||||||
'απόγευμα, οπότε όποτε σε βολεύει εμένα με βολεύει. Ευχαριστώ ξανά που το ετοίμασες, ξέρω πως ' +
|
|
||||||
'πήρε χρόνο.',
|
|
||||||
uk:
|
|
||||||
'Привіт, вибач за пізню відповідь — я весь ранок сидів на нараді і щойно звідти вийшов. Подивився ' +
|
|
||||||
'документ, який ти надіслав, і чесно кажучи, мої думки такі — майже все добре. У мене є пара ' +
|
|
||||||
'речей, які я б змінив у другій половині, але нічого серйозного. Вступ написаний дуже добре. ' +
|
|
||||||
'Напиши, коли матимеш кілька хвилин, і спокійно все обговоримо. Я вільний увесь день, тобто коли тобі зручно, ' +
|
|
||||||
'мені теж підходить. Дякую ще раз, що зібрав це докупи, знаю, що це зайняло час.',
|
|
||||||
ru:
|
|
||||||
'Привет, извини, что долго не отвечал — всё утро просидел на встрече и только что освободился. ' +
|
|
||||||
'Посмотрел документ, который ты прислал, и, честно говоря, думаю, что почти всё нормально. Есть ' +
|
|
||||||
'пара моментов, которые я бы изменил во второй половине, но ничего серьёзного. Введение написано ' +
|
|
||||||
'правда хорошо. Напиши, когда будет пара свободных минут, спокойно всё обсудим. Я свободен весь ' +
|
|
||||||
'день, так что когда тебе удобно, мне тоже подходит. Спасибо ещё раз, что всё это собрал, знаю, ' +
|
|
||||||
'что это заняло время.',
|
|
||||||
ro:
|
|
||||||
'Salut, scuze pentru răspunsul întârziat — am fost toată dimineața într-o ședință și abia acum am ' +
|
|
||||||
'scăpat. M-am uitat pe documentul pe care mi l-ai trimis și, sincer, cred că e aproape totul ' +
|
|
||||||
'bine. Sunt vreo două-trei lucruri pe care le-aș schimba în a doua jumătate, dar nimic grav. ' +
|
|
||||||
'Introducerea e scrisă chiar bine. Spune-mi când ai câteva minute și vorbim pe îndelete despre ' +
|
|
||||||
'asta. Sunt liber toată după-amiaza, deci oricând îți convine mie îmi convine. Mulțumesc încă o ' +
|
|
||||||
'dată că ai pus totul cap la cap, știu că a durat.',
|
|
||||||
hu:
|
|
||||||
'Szia, bocs a késői válaszért — egész délelőtt gyűlésen ültem, és csak most sikerült ' +
|
|
||||||
'kiszabadulnom. Megnéztem a dokumentumot, amit küldtél, és őszintén szólva szerintem szinte ' +
|
|
||||||
'minden rendben van. Van pár dolog, amit megváltoztatnék a második felében, de semmi komoly. A ' +
|
|
||||||
'bevezető nagyon jól van megírva. Szólj, ha van pár szabad perced, és nyugodtan átbeszéljük. ' +
|
|
||||||
'Egész délután érek rá, szóval amikor neked jó, nekem is megfelel. Köszönöm még egyszer, hogy ' +
|
|
||||||
'összeraktad, tudom, hogy hosszú időbe telt.',
|
|
||||||
ja:
|
|
||||||
'返信が遅くなってごめん。朝からずっと会議で、今やっと抜けられたところ。送ってもらった資料、見たよ。' +
|
|
||||||
'正直、ほとんど問題ないと思う。後半のところで直したい部分が二つ三つあるけど、大したことじゃない。' +
|
|
||||||
'導入の部分はすごくよく書けてると思う。時間があるときに教えて、ゆっくり話そう。' +
|
|
||||||
'午後はずっと空いてるから、都合のいいときでいいよ。まとめてくれてありがとう、時間かかったよね。',
|
|
||||||
zh:
|
|
||||||
'抱歉这么晚才回你——我一早上都在开会,刚出来。你发的那份文件我看过了,说实话,大部分都挺好的。' +
|
|
||||||
'后半部分有几个地方我想改一下,但都不是什么大问题。开头写得真的很不错。你什么时候有空跟我说一声,我们好好聊聊。' +
|
|
||||||
'我整个下午都有空,所以你什么时候方便都行。再次谢谢你把这个整理出来,我知道挺花时间的。',
|
|
||||||
ko:
|
|
||||||
'답장이 늦어서 미안해. 오전 내내 회의하다가 방금 겨우 빠져나왔어. 보내준 문서 봤는데, 솔직히 거의 다 괜찮은 것 같아. 후반부에 고치고 싶은 부분이 두어 개 있긴 한데, ' +
|
|
||||||
'심각한 건 아니야. 서론은 진짜 잘 쓴 것 같아. 시간 될 때 얘기해줘, 그럼 차분히 얘기해보자. 오후 내내 시간 있으니까 편한 때 아무 때나 괜찮아. 정리해줘서 다시 한번 ' +
|
|
||||||
'고마워, 시간 좀 걸렸을 텐데.',
|
|
||||||
ar:
|
|
||||||
'مرحبًا، آسف على تأخري في الرد — كنت في اجتماع طوال الصباح ولتوّي خرجت منه. اطّلعت على المستند ' +
|
|
||||||
'الذي أرسلته، وبصراحة أعتقد أن معظمه جيد. هناك بعض الأشياء التي أود تغييرها في النصف الثاني، لكن ' +
|
|
||||||
'لا شيء خطير. المقدمة مكتوبة بشكل جيد جدًا. هل يمكن أن تخبرني متى تكون لديك بضع دقائق لنتحدث ' +
|
|
||||||
'بهدوء؟ أنا متفرغ طوال بعد الظهر، فأي وقت يناسبك يناسبني. شكرًا مرة أخرى على إعدادك لهذا، أعرف أن ' +
|
|
||||||
'الأمر استغرق وقتًا.',
|
|
||||||
he:
|
|
||||||
'היי, סליחה על התגובה המאוחרת — הייתי בישיבה כל הבוקר ורק עכשיו יצאתי. הסתכלתי על המסמך ששלחת, ' +
|
|
||||||
'ובכנות, אני חושב שרוב הדברים בסדר. יש כמה דברים שהייתי משנה בחצי השני, אבל שום דבר רציני. הפתיחה ' +
|
|
||||||
'כתובה ממש טוב. תגיד לי מתי יש לך כמה דקות ונדבר על זה בנחת. אני פנוי כל אחר הצהריים, אז מתי שנוח ' +
|
|
||||||
'לך נוח גם לי. תודה שוב שסידרת את זה, אני יודע שזה לקח זמן.',
|
|
||||||
hi:
|
|
||||||
'हाय, देर से जवाब देने के लिए माफ़ करना — मैं पूरी सुबह मीटिंग में था और अभी-अभी निकल पाया। तुमने ' +
|
|
||||||
'जो डॉक्यूमेंट भेजा था, वो देख लिया, और सच कहूं तो ज़्यादातर हिस्सा ठीक लग रहा है। दूसरे हिस्से ' +
|
|
||||||
'में कुछ चीज़ें हैं जो मैं बदलना चाहूंगा, पर कुछ गंभीर नहीं है। शुरुआत बहुत अच्छी लिखी गई है। जब ' +
|
|
||||||
'भी कुछ मिनट मिलें बता देना, फिर आराम से बात करते हैं। पूरी दोपहर मैं फ्री हूं, तो जब भी तुम्हें ' +
|
|
||||||
'सही लगे मुझे ठीक है। इसे तैयार करने के लिए फिर से शुक्रिया, पता है इसमें वक़्त लगा होगा।',
|
|
||||||
};
|
|
||||||
|
|
||||||
// Beam search rather than the default greedy decode (whisper.cpp ships `beam_size = -1`). It is the
|
|
||||||
// single biggest quality lever here and it costs time, which this flow has — nothing waits on it.
|
|
||||||
const BEAM_SIZE = 5;
|
|
||||||
const BEST_OF = 5;
|
|
||||||
|
|
||||||
// The whisper container already runs silero VAD, but its defaults assume clean dictation. Widen the
|
|
||||||
// segments so they break at real pauses instead of at breaths, and pad the edges so word onsets are
|
|
||||||
// not clipped — clipped onsets are what shows up in the output as a word split down the middle.
|
|
||||||
const VAD_TUNING: Record<string, string> = {
|
|
||||||
vad_speech_pad_ms: '200', // default 30
|
|
||||||
vad_min_silence_duration_ms: '500', // default 100 — a breath is not a sentence boundary
|
|
||||||
vad_samples_overlap: '0.2', // default 0.1
|
|
||||||
vad_max_speech_duration_s: '25', // default unbounded — keep a segment inside one 30s window
|
|
||||||
};
|
|
||||||
|
|
||||||
// A silence this long reads as a new thought. Whisper reports segment times against the original
|
|
||||||
// audio even when VAD is on (whisper.cpp maps them back through its VAD table), so these really are
|
|
||||||
// gaps in the recording rather than artefacts of the filtering.
|
|
||||||
const PARAGRAPH_GAP_SECONDS = 1.2;
|
|
||||||
const MAX_PARAGRAPH_CHARS = 900;
|
|
||||||
|
|
||||||
// OpenAI Whisper's own silence rule: a segment the model thinks is probably not speech *and* decodes
|
|
||||||
// with low confidence is a hallucination rather than a transcript. whisper.cpp applies a subtly
|
|
||||||
// different form of this internally, so it is worth re-applying to the values it reports — this is
|
|
||||||
// what removes the stray words that appear over music and room tone.
|
|
||||||
const NO_SPEECH_PROB = 0.6;
|
|
||||||
const LOW_CONFIDENCE_LOGPROB = -1.0;
|
|
||||||
|
|
||||||
export type WhisperSegment = {
|
|
||||||
text?: string;
|
|
||||||
start?: number;
|
|
||||||
end?: number;
|
|
||||||
no_speech_prob?: number;
|
|
||||||
avg_logprob?: number;
|
|
||||||
};
|
|
||||||
|
|
||||||
// Defaults are chosen so a segment missing either field is kept: dropping real speech is much worse
|
|
||||||
// than keeping a stray word.
|
|
||||||
const isHallucinatedSilence = (segment: WhisperSegment) =>
|
|
||||||
(segment.no_speech_prob ?? 0) > NO_SPEECH_PROB && (segment.avg_logprob ?? 0) < LOW_CONFIDENCE_LOGPROB;
|
|
||||||
|
|
||||||
// Whisper emits each segment with a leading space and now and then strands trailing punctuation in a
|
|
||||||
// segment of its own, which is why the old `response_format=text` output contained lone "." lines.
|
|
||||||
const appendSegment = (paragraph: string, text: string) => {
|
|
||||||
if (!paragraph) return text;
|
|
||||||
return /^[.,;:!?…)\]]/.test(text) ? paragraph + text : `${paragraph} ${text}`;
|
|
||||||
};
|
|
||||||
|
|
||||||
// Whisper returns one segment per utterance, not per paragraph. Rejoin them into prose and start a
|
|
||||||
// new paragraph on a long pause, so a voice memo reads as something written rather than as a list of
|
|
||||||
// fragments.
|
|
||||||
export function toParagraphs(segments: WhisperSegment[]): string {
|
|
||||||
const paragraphs: string[] = [];
|
|
||||||
let current = '';
|
|
||||||
let previousEnd: number | null = null;
|
|
||||||
|
|
||||||
for (const segment of segments) {
|
|
||||||
const text = (segment.text ?? '').replace(/\s+/g, ' ').trim();
|
|
||||||
if (!text || isHallucinatedSilence(segment)) continue;
|
|
||||||
|
|
||||||
const gap = previousEnd !== null && segment.start !== undefined ? segment.start - previousEnd : 0;
|
|
||||||
// Only break an over-long paragraph where a sentence actually ended, never mid-thought.
|
|
||||||
const overlong = current.length >= MAX_PARAGRAPH_CHARS && /[.!?…]["')\]]?$/.test(current);
|
|
||||||
|
|
||||||
if (current && (gap >= PARAGRAPH_GAP_SECONDS || overlong)) {
|
|
||||||
paragraphs.push(current);
|
|
||||||
current = '';
|
|
||||||
}
|
|
||||||
|
|
||||||
current = appendSegment(current, text);
|
|
||||||
if (segment.end !== undefined) previousEnd = segment.end;
|
|
||||||
}
|
|
||||||
|
|
||||||
if (current) paragraphs.push(current);
|
|
||||||
return paragraphs.join('\n\n');
|
|
||||||
}
|
|
||||||
|
|
||||||
export async function transcribeAudio({
|
|
||||||
file,
|
|
||||||
whisperUrl,
|
|
||||||
spokenLanguages,
|
|
||||||
}: TranscribeArgs): Promise<TranscribeResult> {
|
|
||||||
const base = whisperUrl.replace(/\/+$/, '');
|
|
||||||
|
|
||||||
// Step 1: detect language, and only fall back to the spoken-language bias when Whisper is unsure.
|
|
||||||
const detectForm = new FormData();
|
|
||||||
detectForm.append('file', file);
|
|
||||||
detectForm.append('temperature', '0.0');
|
|
||||||
detectForm.append('response_format', 'verbose_json');
|
|
||||||
detectForm.append('detect_language', 'true');
|
|
||||||
|
|
||||||
const detectRes = await fetch(`${base}/inference`, {
|
|
||||||
method: 'POST',
|
|
||||||
body: detectForm,
|
|
||||||
timeout: false,
|
|
||||||
} as BunRequestInit);
|
|
||||||
if (!detectRes.ok) throw new Error(`Language detection failed (${detectRes.status})`);
|
|
||||||
const detectJson = (await detectRes.json()) as {
|
|
||||||
language?: string;
|
|
||||||
detected_language?: string;
|
|
||||||
detected_language_probability?: number;
|
|
||||||
language_probabilities?: Record<string, number>;
|
|
||||||
};
|
|
||||||
|
|
||||||
const probs = detectJson.language_probabilities;
|
|
||||||
const ranked = probs ? Object.entries(probs).sort((a, b) => b[1] - a[1]) : [];
|
|
||||||
const [topCode, topProb] = ranked[0] ?? [];
|
|
||||||
|
|
||||||
let detectedLanguage: string;
|
|
||||||
if (topCode && (topProb ?? 0) >= CONFIDENT_DETECTION) {
|
|
||||||
// Whisper is sure. Trust it even when it is not a language the user speaks — that is exactly the
|
|
||||||
// case translation exists for.
|
|
||||||
detectedLanguage = topCode;
|
|
||||||
} else if (ranked.length > 0 && spokenLanguages.length > 0) {
|
|
||||||
// Genuinely ambiguous (usually a short clip). Break the tie toward a language the user speaks,
|
|
||||||
// since Whisper's low-confidence guess defaults to English regardless of the audio.
|
|
||||||
const bestSpoken = spokenLanguages
|
|
||||||
.map((code) => [code, probs?.[code] ?? 0] as const)
|
|
||||||
.sort((a, b) => b[1] - a[1])[0];
|
|
||||||
detectedLanguage = bestSpoken?.[0] ?? topCode ?? 'en';
|
|
||||||
} else if (topCode) {
|
|
||||||
detectedLanguage = topCode;
|
|
||||||
} else {
|
|
||||||
// No probabilities field (non-whisper.cpp backend) — try to map the reported name.
|
|
||||||
const raw = (detectJson.detected_language ?? detectJson.language ?? 'en').toLowerCase();
|
|
||||||
detectedLanguage = LANG_NAME_TO_ISO[raw] ?? raw;
|
|
||||||
}
|
|
||||||
|
|
||||||
// Step 2: transcribe in the detected language
|
|
||||||
const transcribeForm = new FormData();
|
|
||||||
transcribeForm.append('file', file);
|
|
||||||
transcribeForm.append('language', detectedLanguage);
|
|
||||||
transcribeForm.append('temperature', '0.0');
|
|
||||||
transcribeForm.append('temperature_inc', '0.2');
|
|
||||||
transcribeForm.append('beam_size', String(BEAM_SIZE));
|
|
||||||
transcribeForm.append('best_of', String(BEST_OF));
|
|
||||||
transcribeForm.append('suppress_nst', 'true');
|
|
||||||
transcribeForm.append('response_format', 'verbose_json');
|
|
||||||
// verbose_json re-runs a full language auto-detect unless this is set. Step 1 already answered
|
|
||||||
// that question, so it is pure duplicated work on every transcription.
|
|
||||||
transcribeForm.append('no_language_probabilities', 'true');
|
|
||||||
// Segment timestamps drive the paragraphing below and are kept; per-token timestamps are extra
|
|
||||||
// work on every segment and nothing here reads them.
|
|
||||||
transcribeForm.append('token_timestamps', 'false');
|
|
||||||
|
|
||||||
const prompt = TRANSCRIPT_PROMPTS[detectedLanguage];
|
|
||||||
if (prompt) {
|
|
||||||
transcribeForm.append('prompt', prompt);
|
|
||||||
transcribeForm.append('carry_initial_prompt', 'true');
|
|
||||||
transcribeForm.append('max_context', String(PROMPT_CONTEXT_TOKENS));
|
|
||||||
} else {
|
|
||||||
// Nothing worth protecting the budget for, so close the loop-feeding path outright.
|
|
||||||
transcribeForm.append('max_context', '0');
|
|
||||||
}
|
|
||||||
|
|
||||||
for (const [key, value] of Object.entries(VAD_TUNING)) transcribeForm.append(key, value);
|
|
||||||
|
|
||||||
const res = await fetch(`${base}/inference`, {
|
|
||||||
method: 'POST',
|
|
||||||
body: transcribeForm,
|
|
||||||
timeout: false,
|
|
||||||
} as BunRequestInit);
|
|
||||||
if (!res.ok) throw new Error(`Transcription failed (${res.status})`);
|
|
||||||
|
|
||||||
const json = (await res.json()) as { text?: string; segments?: WhisperSegment[] };
|
|
||||||
// Fall back to the flat text for any backend that does not return segments.
|
|
||||||
const segments = json.segments ?? [];
|
|
||||||
const text = segments.length > 0 ? toParagraphs(segments) : (json.text ?? '').trim();
|
|
||||||
|
|
||||||
return { text, detectedLanguage };
|
|
||||||
}
|
|
||||||
@@ -14,14 +14,15 @@ const PUBLIC_HOST = (() => {
|
|||||||
type TaskApiUser = { id: number; email: string; username?: string | null };
|
type TaskApiUser = { id: number; email: string; username?: string | null };
|
||||||
|
|
||||||
// Environment that lets a script task call Officer's own API instead of reimplementing server-side
|
// Environment that lets a script task call Officer's own API instead of reimplementing server-side
|
||||||
// work (transcription, for one — the endpoint already handles language detection and caching).
|
// work (transcription, for one — `plugins/transcribe/tasks/transcribe-audio-file/index.ts` uses exactly
|
||||||
|
// this to reach the transcribe plugin's own route rather than talking to Whisper itself).
|
||||||
//
|
//
|
||||||
// Requests go to 127.0.0.1 so a task never depends on DNS or the reverse proxy being up. Origin
|
// Requests go to 127.0.0.1 so a task never depends on DNS or the reverse proxy being up. Origin
|
||||||
// validation matches the Host header against PUBLIC_URL though, so a bare localhost request is
|
// validation matches the Host header against PUBLIC_URL though, so a bare localhost request is
|
||||||
// rejected — hence OFFICER_API_HOST, which scripts must pass as an explicit Host header:
|
// rejected — hence OFFICER_API_HOST, which scripts must pass as an explicit Host header:
|
||||||
//
|
//
|
||||||
// curl -H "Authorization: Bearer $OFFICER_AUTH_TOKEN" -H "Host: $OFFICER_API_HOST" \
|
// curl -H "Authorization: Bearer $OFFICER_AUTH_TOKEN" -H "Host: $OFFICER_API_HOST" \
|
||||||
// "$OFFICER_API_URL/api/file-browser/transcribe"
|
// "$OFFICER_API_URL/api/transcribe/transcribe"
|
||||||
export async function buildTaskApiEnv(user: TaskApiUser): Promise<Record<string, string>> {
|
export async function buildTaskApiEnv(user: TaskApiUser): Promise<Record<string, string>> {
|
||||||
const token = await sign({ id: user.id, email: user.email, username: user.username ?? '' }, '12h');
|
const token = await sign({ id: user.id, email: user.email, username: user.username ?? '' }, '12h');
|
||||||
return {
|
return {
|
||||||
|
|||||||
@@ -5,9 +5,11 @@
|
|||||||
//
|
//
|
||||||
// The obvious model is "list the routes a role may call". It does not survive contact with this codebase.
|
// The obvious model is "list the routes a role may call". It does not survive contact with this codebase.
|
||||||
// A sweep of all 100 mutating platform routes on 2026-08-06 found reads permanently stuck on POST for two
|
// A sweep of all 100 mutating platform routes on 2026-08-06 found reads permanently stuck on POST for two
|
||||||
// reasons that are not going away: bodies GET cannot carry (`/stt` multipart audio, `/tts`, `/ocr`,
|
// reasons that are not going away: bodies GET cannot carry (`/tts`, `/ocr` multipart audio and images —
|
||||||
// `/transcribe`), and credentials that must not sit in a query string where access logs, shell history and
|
// `/stt` and `/transcribe` were two more of these until 2026-08-24, when speech-to-text left for the
|
||||||
// Referer headers capture them (`/tts/voices` apiKey, the four `/test` endpoints, `/local-providers/probe`).
|
// `transcribe` plugin), and credentials that must not sit in a query string where access logs, shell
|
||||||
|
// history and Referer headers capture them (`/tts/voices` apiKey, the four `/test` endpoints,
|
||||||
|
// `/local-providers/probe`).
|
||||||
// Five genuinely free conversions were done in e54d71d; the rest are staying. So the METHOD alone cannot
|
// Five genuinely free conversions were done in e54d71d; the rest are staying. So the METHOD alone cannot
|
||||||
// carry the read/write distinction — hence `readOnlyWrites` below, declared per permission.
|
// carry the read/write distinction — hence `readOnlyWrites` below, declared per permission.
|
||||||
//
|
//
|
||||||
|
|||||||
@@ -311,6 +311,27 @@ const BUNDLED: CatalogueEntry[] = [
|
|||||||
// No schema. The session is processes and a wine prefix on disk, and none of that belongs in Postgres.
|
// No schema. The session is processes and a wine prefix on disk, and none of that belongs in Postgres.
|
||||||
has: { api: true, schema: false, sidecar: true, web: true },
|
has: { api: true, schema: false, sidecar: true, web: true },
|
||||||
},
|
},
|
||||||
|
{
|
||||||
|
appName: 'transcribe',
|
||||||
|
publisher: 'officerdev',
|
||||||
|
version: '1.0.0',
|
||||||
|
platform: '>=1.0.0',
|
||||||
|
label: 'Transcribe',
|
||||||
|
summary: 'Speech-to-text over your own whisper.cpp server',
|
||||||
|
color: '#f97316',
|
||||||
|
icon: 'AudioLines',
|
||||||
|
source: 'https://gitea.officer.dev/plugins/transcribe.git',
|
||||||
|
permissions: [
|
||||||
|
{
|
||||||
|
key: 'transcribe',
|
||||||
|
label: 'Transcribe',
|
||||||
|
description: 'Turn audio into text over your own whisper.cpp server',
|
||||||
|
},
|
||||||
|
],
|
||||||
|
// No web/. A backend-only plugin has no screen to pin, which `plugin.web` already gates — this one is
|
||||||
|
// reached through the file browser and by whatever else asks the platform to turn audio into text.
|
||||||
|
has: { api: true, schema: false, sidecar: true, web: false },
|
||||||
|
},
|
||||||
{
|
{
|
||||||
appName: 'jellyfin',
|
appName: 'jellyfin',
|
||||||
publisher: 'officerdev',
|
publisher: 'officerdev',
|
||||||
|
|||||||
@@ -20,7 +20,6 @@ export { EmbeddableChat, type UseEmbeddableChatType } from './EmbeddableChat';
|
|||||||
export { useChat, type UseChatType } from '../../hooks/useChat';
|
export { useChat, type UseChatType } from '../../hooks/useChat';
|
||||||
export { useSlashCommands } from './useSlashCommands';
|
export { useSlashCommands } from './useSlashCommands';
|
||||||
export { useAttachments, type UseAttachmentsType } from './useAttachments';
|
export { useAttachments, type UseAttachmentsType } from './useAttachments';
|
||||||
export { useAudioRecording, type UseAudioRecordingType } from './useAudioRecording';
|
|
||||||
|
|
||||||
export * from './types';
|
export * from './types';
|
||||||
|
|
||||||
|
|||||||
@@ -1,83 +0,0 @@
|
|||||||
import { afterEach, describe, expect, test } from 'bun:test';
|
|
||||||
import { act, cleanup, render } from '@testing-library/react';
|
|
||||||
import { useAudioRecording } from './useAudioRecording';
|
|
||||||
|
|
||||||
/**
|
|
||||||
* The microphone is the one resource in this app the browser tells the user about: a recording indicator
|
|
||||||
* stays lit in the tab strip until every track is stopped. Nothing here released it on unmount, so closing
|
|
||||||
* a Chat panel mid-recording left the light on for the life of the tab with no way to switch it off.
|
|
||||||
*
|
|
||||||
* The stubs are the smallest thing that can answer "was the track stopped": a stream that counts stops and
|
|
||||||
* a recorder that reports its state. Neither the real MediaRecorder nor a real device is involved.
|
|
||||||
*/
|
|
||||||
|
|
||||||
const stopped: string[] = [];
|
|
||||||
|
|
||||||
const fakeStream = (label: string) => ({
|
|
||||||
getTracks: () => [{ stop: () => stopped.push(label) }],
|
|
||||||
});
|
|
||||||
|
|
||||||
class FakeMediaRecorder {
|
|
||||||
state: 'recording' | 'inactive' = 'inactive';
|
|
||||||
stream = fakeStream('track');
|
|
||||||
mimeType = 'audio/webm';
|
|
||||||
ondataavailable: ((ev: { data: Blob }) => void) | null = null;
|
|
||||||
onstop: (() => void) | null = null;
|
|
||||||
start() {
|
|
||||||
this.state = 'recording';
|
|
||||||
}
|
|
||||||
stop() {
|
|
||||||
this.state = 'inactive';
|
|
||||||
this.onstop?.();
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
const install = () => {
|
|
||||||
stopped.length = 0;
|
|
||||||
Object.assign(globalThis, { MediaRecorder: FakeMediaRecorder });
|
|
||||||
Object.defineProperty(globalThis.navigator, 'mediaDevices', {
|
|
||||||
configurable: true,
|
|
||||||
value: { getUserMedia: async () => fakeStream('track') },
|
|
||||||
});
|
|
||||||
};
|
|
||||||
|
|
||||||
type Handle = { toggle: () => Promise<void>; recording: boolean };
|
|
||||||
|
|
||||||
const handle: Handle = { toggle: async () => {}, recording: false };
|
|
||||||
|
|
||||||
const Probe = () => {
|
|
||||||
const runner = useAudioRecording(() => {});
|
|
||||||
handle.toggle = runner.toggleRecording;
|
|
||||||
handle.recording = runner.recording;
|
|
||||||
return null;
|
|
||||||
};
|
|
||||||
|
|
||||||
afterEach(cleanup);
|
|
||||||
|
|
||||||
describe('useAudioRecording', () => {
|
|
||||||
test('releases the microphone when its component goes away mid-recording', async () => {
|
|
||||||
install();
|
|
||||||
const view = render(<Probe />);
|
|
||||||
|
|
||||||
await act(async () => {
|
|
||||||
await handle.toggle();
|
|
||||||
});
|
|
||||||
expect(handle.recording).toBe(true);
|
|
||||||
expect(stopped).toEqual([]);
|
|
||||||
|
|
||||||
view.unmount();
|
|
||||||
|
|
||||||
// Not "the user pressed stop" — the panel was closed, swapped or remounted while still recording, and
|
|
||||||
// that is the case that used to leak.
|
|
||||||
expect(stopped).toEqual(['track']);
|
|
||||||
});
|
|
||||||
|
|
||||||
test('unmounting when nothing was ever recorded does nothing', () => {
|
|
||||||
install();
|
|
||||||
const view = render(<Probe />);
|
|
||||||
|
|
||||||
view.unmount();
|
|
||||||
|
|
||||||
expect(stopped).toEqual([]);
|
|
||||||
});
|
|
||||||
});
|
|
||||||
@@ -1,114 +1,7 @@
|
|||||||
import { useState, useRef, useEffect } from 'react';
|
// The chat composer's own STT-driven recording hook lived here until 2026-08-24, when speech-to-text
|
||||||
import { toast } from 'sonner';
|
// left the platform for the `transcribe` plugin. It had no caller — the composer never mounted it — so
|
||||||
import { useTranslation } from '../../locales';
|
// it went rather than being ported. `blobToWav` stays: `DictateDialog.tsx` in the file browser uses it
|
||||||
|
// for something that was never STT, just recording a voice memo straight to a `.wav` upload.
|
||||||
export function useAudioRecording(onTranscription: (text: string) => void) {
|
|
||||||
const { t } = useTranslation();
|
|
||||||
const [recording, setRecording] = useState(false);
|
|
||||||
const [transcribing, setTranscribing] = useState(false);
|
|
||||||
const mediaRecorderRef = useRef<MediaRecorder | null>(null);
|
|
||||||
const chunksRef = useRef<Blob[]>([]);
|
|
||||||
|
|
||||||
const toggleRecording = async () => {
|
|
||||||
if (recording) {
|
|
||||||
const recorder = mediaRecorderRef.current;
|
|
||||||
if (!recorder) return;
|
|
||||||
|
|
||||||
setRecording(false);
|
|
||||||
|
|
||||||
try {
|
|
||||||
if (recorder.state === 'inactive') {
|
|
||||||
recorder.stream.getTracks().forEach((track) => track.stop());
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
const blob = await new Promise<Blob>((resolve, reject) => {
|
|
||||||
const timeout = setTimeout(() => reject(new Error('Recording stop timed out')), 5000);
|
|
||||||
recorder.onstop = () => {
|
|
||||||
clearTimeout(timeout);
|
|
||||||
resolve(new Blob(chunksRef.current, { type: recorder.mimeType }));
|
|
||||||
chunksRef.current = [];
|
|
||||||
};
|
|
||||||
recorder.stop();
|
|
||||||
});
|
|
||||||
|
|
||||||
recorder.stream.getTracks().forEach((track) => track.stop());
|
|
||||||
|
|
||||||
if (blob.size === 0) {
|
|
||||||
toast.error(t('chat.composer.noAudio'));
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
setTranscribing(true);
|
|
||||||
try {
|
|
||||||
const wav = await blobToWav(blob);
|
|
||||||
const formData = new FormData();
|
|
||||||
formData.append('file', wav, 'recording.wav');
|
|
||||||
formData.append('temperature', '0.0');
|
|
||||||
formData.append('temperature_inc', '0.2');
|
|
||||||
formData.append('response_format', 'json');
|
|
||||||
|
|
||||||
const token = localStorage.getItem('BEARER_TOKEN') ?? sessionStorage.getItem('BEARER_TOKEN');
|
|
||||||
const res = await fetch('/api/chat/stt', {
|
|
||||||
method: 'POST',
|
|
||||||
body: formData,
|
|
||||||
headers: token ? { Authorization: `Bearer ${token}` } : {},
|
|
||||||
});
|
|
||||||
if (!res.ok) throw new Error(`Whisper returned ${res.status}`);
|
|
||||||
const json = await res.json();
|
|
||||||
if (json.error) throw new Error(json.error);
|
|
||||||
const text = (json.text ?? '').trim();
|
|
||||||
if (text) onTranscription(text);
|
|
||||||
} finally {
|
|
||||||
setTranscribing(false);
|
|
||||||
}
|
|
||||||
} catch (err) {
|
|
||||||
recorder.stream?.getTracks().forEach((track) => track.stop());
|
|
||||||
chunksRef.current = [];
|
|
||||||
toast.error(err instanceof Error ? err.message : 'Recording failed');
|
|
||||||
}
|
|
||||||
return;
|
|
||||||
}
|
|
||||||
|
|
||||||
try {
|
|
||||||
const stream = await navigator.mediaDevices.getUserMedia({ audio: true });
|
|
||||||
const recorder = new MediaRecorder(stream);
|
|
||||||
mediaRecorderRef.current = recorder;
|
|
||||||
chunksRef.current = [];
|
|
||||||
|
|
||||||
recorder.ondataavailable = (ev) => {
|
|
||||||
if (ev.data.size > 0) chunksRef.current.push(ev.data);
|
|
||||||
};
|
|
||||||
|
|
||||||
recorder.start(250);
|
|
||||||
setRecording(true);
|
|
||||||
} catch (err) {
|
|
||||||
toast.error(err instanceof Error ? err.message : 'Could not access microphone');
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
// Nothing here released the microphone on unmount, and a panel can go away mid-recording — closing the
|
|
||||||
// Chat panel, switching dashboards, a layout change that remounts the slot. The recorder is only ever
|
|
||||||
// stopped by the user pressing the button again, so without this the stream outlives the component and
|
|
||||||
// the browser keeps showing the recording indicator for the life of the tab.
|
|
||||||
useEffect(
|
|
||||||
() => () => {
|
|
||||||
const recorder = mediaRecorderRef.current;
|
|
||||||
if (!recorder) return;
|
|
||||||
if (recorder.state !== 'inactive') recorder.stop();
|
|
||||||
recorder.stream?.getTracks().forEach((track) => track.stop());
|
|
||||||
mediaRecorderRef.current = null;
|
|
||||||
chunksRef.current = [];
|
|
||||||
},
|
|
||||||
[],
|
|
||||||
);
|
|
||||||
|
|
||||||
return { recording, transcribing, toggleRecording };
|
|
||||||
}
|
|
||||||
|
|
||||||
export type UseAudioRecordingType = ReturnType<typeof useAudioRecording>;
|
|
||||||
|
|
||||||
// ── Helpers ──
|
|
||||||
|
|
||||||
export const blobToWav = async (blob: Blob): Promise<Blob> => {
|
export const blobToWav = async (blob: Blob): Promise<Blob> => {
|
||||||
const ctx = new AudioContext();
|
const ctx = new AudioContext();
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ export type FileViewerContextValue = {
|
|||||||
directContent: boolean;
|
directContent: boolean;
|
||||||
ttsLoading: boolean;
|
ttsLoading: boolean;
|
||||||
ocrLoading: boolean;
|
ocrLoading: boolean;
|
||||||
transcribeLoading: boolean;
|
|
||||||
extractAudioLoading: boolean;
|
extractAudioLoading: boolean;
|
||||||
extractLoading: boolean;
|
extractLoading: boolean;
|
||||||
autoPlay: boolean;
|
autoPlay: boolean;
|
||||||
@@ -22,7 +21,6 @@ export type FileViewerContextValue = {
|
|||||||
setContent: (content: string) => void;
|
setContent: (content: string) => void;
|
||||||
handleReadAloud: () => void;
|
handleReadAloud: () => void;
|
||||||
handleOcr: () => void;
|
handleOcr: () => void;
|
||||||
handleTranscribe: () => void;
|
|
||||||
handleExtractAudio: () => void;
|
handleExtractAudio: () => void;
|
||||||
handleExtract: () => void;
|
handleExtract: () => void;
|
||||||
handleDownload: () => void;
|
handleDownload: () => void;
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ import {
|
|||||||
FileType2,
|
FileType2,
|
||||||
Volume2,
|
Volume2,
|
||||||
ScanText,
|
ScanText,
|
||||||
FileText,
|
|
||||||
AudioLines,
|
AudioLines,
|
||||||
FolderArchive,
|
FolderArchive,
|
||||||
Save,
|
Save,
|
||||||
@@ -28,12 +27,10 @@ export const FileViewerHeader = () => {
|
|||||||
directContent,
|
directContent,
|
||||||
ttsLoading,
|
ttsLoading,
|
||||||
ocrLoading,
|
ocrLoading,
|
||||||
transcribeLoading,
|
|
||||||
extractAudioLoading,
|
extractAudioLoading,
|
||||||
extractLoading,
|
extractLoading,
|
||||||
handleReadAloud,
|
handleReadAloud,
|
||||||
handleOcr,
|
handleOcr,
|
||||||
handleTranscribe,
|
|
||||||
handleExtractAudio,
|
handleExtractAudio,
|
||||||
handleExtract,
|
handleExtract,
|
||||||
handleDownload,
|
handleDownload,
|
||||||
@@ -98,31 +95,6 @@ export const FileViewerHeader = () => {
|
|||||||
{ocrLoading ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <ScanText className="h-3.5 w-3.5" />}
|
{ocrLoading ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <ScanText className="h-3.5 w-3.5" />}
|
||||||
</button>
|
</button>
|
||||||
)}
|
)}
|
||||||
{/*
|
|
||||||
TRANSCRIBE — hidden, not deleted, and the distinction is deliberate.
|
|
||||||
|
|
||||||
Speech-to-text is leaving the platform for a plugin of its own. Until that lands this button calls
|
|
||||||
`POST /file-browser/transcribe`, which refuses on every server here because no STT endpoint is
|
|
||||||
configured anywhere — so it is a control that can only fail, and one of those teaches people to
|
|
||||||
distrust the controls beside it.
|
|
||||||
|
|
||||||
Kept because the wiring is the useful part. When the plugin exists this becomes a call to it, and
|
|
||||||
the handler, the loading state and the sibling-markdown behaviour in FileViewerProvider are all
|
|
||||||
still correct; deleting would mean rediscovering them.
|
|
||||||
|
|
||||||
It may not come back here at all. The other route to the same job is the file browser's context
|
|
||||||
menu, which a plugin reaches with a task — and that path needs no button in this header.
|
|
||||||
*/}
|
|
||||||
{false && fileType === 'audio' && (
|
|
||||||
<button
|
|
||||||
onClick={handleTranscribe}
|
|
||||||
disabled={transcribeLoading}
|
|
||||||
className="p-1 rounded hover:bg-current/10 disabled:opacity-40 transition-colors cursor-pointer"
|
|
||||||
title={t('fileViewer.transcribe')}
|
|
||||||
>
|
|
||||||
{transcribeLoading ? <Loader2 className="h-3.5 w-3.5 animate-spin" /> : <FileText className="h-3.5 w-3.5" />}
|
|
||||||
</button>
|
|
||||||
)}
|
|
||||||
{fileType === 'video' && (
|
{fileType === 'video' && (
|
||||||
<button
|
<button
|
||||||
onClick={handleExtractAudio}
|
onClick={handleExtractAudio}
|
||||||
|
|||||||
@@ -55,7 +55,6 @@ export const FileViewerProvider = ({
|
|||||||
const [error, setError] = useState<string | null>(null);
|
const [error, setError] = useState<string | null>(null);
|
||||||
const [ttsLoading, setTtsLoading] = useState(false);
|
const [ttsLoading, setTtsLoading] = useState(false);
|
||||||
const [ocrLoading, setOcrLoading] = useState(false);
|
const [ocrLoading, setOcrLoading] = useState(false);
|
||||||
const [transcribeLoading, setTranscribeLoading] = useState(false);
|
|
||||||
const [extractAudioLoading, setExtractAudioLoading] = useState(false);
|
const [extractAudioLoading, setExtractAudioLoading] = useState(false);
|
||||||
const [extractLoading, setExtractLoading] = useState(false);
|
const [extractLoading, setExtractLoading] = useState(false);
|
||||||
const [saveResultLoading, setSaveResultLoading] = useState(false);
|
const [saveResultLoading, setSaveResultLoading] = useState(false);
|
||||||
@@ -125,19 +124,6 @@ export const FileViewerProvider = ({
|
|||||||
}
|
}
|
||||||
};
|
};
|
||||||
|
|
||||||
const handleTranscribe = async () => {
|
|
||||||
setTranscribeLoading(true);
|
|
||||||
try {
|
|
||||||
const { transcriptionPath, transcriptionRoot } = await files.transcribe(filePath, { saveNextTo: true });
|
|
||||||
bumpFilesRefresh();
|
|
||||||
onReplaceView?.(filePath, root, transcriptionPath, transcriptionRoot);
|
|
||||||
} catch {
|
|
||||||
toast.error(t('fileViewer.transcribeFailed'));
|
|
||||||
} finally {
|
|
||||||
setTranscribeLoading(false);
|
|
||||||
}
|
|
||||||
};
|
|
||||||
|
|
||||||
const handleExtractAudio = async () => {
|
const handleExtractAudio = async () => {
|
||||||
setExtractAudioLoading(true);
|
setExtractAudioLoading(true);
|
||||||
try {
|
try {
|
||||||
@@ -194,7 +180,6 @@ export const FileViewerProvider = ({
|
|||||||
directContent: directContent !== undefined,
|
directContent: directContent !== undefined,
|
||||||
ttsLoading,
|
ttsLoading,
|
||||||
ocrLoading,
|
ocrLoading,
|
||||||
transcribeLoading,
|
|
||||||
extractAudioLoading,
|
extractAudioLoading,
|
||||||
extractLoading,
|
extractLoading,
|
||||||
autoPlay,
|
autoPlay,
|
||||||
@@ -204,7 +189,6 @@ export const FileViewerProvider = ({
|
|||||||
setContent,
|
setContent,
|
||||||
handleReadAloud,
|
handleReadAloud,
|
||||||
handleOcr,
|
handleOcr,
|
||||||
handleTranscribe,
|
|
||||||
handleExtractAudio,
|
handleExtractAudio,
|
||||||
handleExtract,
|
handleExtract,
|
||||||
handleDownload,
|
handleDownload,
|
||||||
|
|||||||
@@ -77,13 +77,6 @@ export const useFilesAPI = (root: string = 'home') => {
|
|||||||
ocr: (path: string, opts?: { saveNextTo?: boolean }) =>
|
ocr: (path: string, opts?: { saveNextTo?: boolean }) =>
|
||||||
client.post<{ ocrPath: string; ocrRoot: string }>('/file-browser/ocr', { path, root, ...opts }),
|
client.post<{ ocrPath: string; ocrRoot: string }>('/file-browser/ocr', { path, root, ...opts }),
|
||||||
|
|
||||||
transcribe: (path: string, opts?: { saveNextTo?: boolean }) =>
|
|
||||||
client.post<{ transcriptionPath: string; transcriptionRoot: string }>('/file-browser/transcribe', {
|
|
||||||
path,
|
|
||||||
root,
|
|
||||||
...opts,
|
|
||||||
}),
|
|
||||||
|
|
||||||
extractAudio: (path: string) =>
|
extractAudio: (path: string) =>
|
||||||
client.post<{ audioPath: string; audioRoot: string }>('/file-browser/extract-audio', { path, root }),
|
client.post<{ audioPath: string; audioRoot: string }>('/file-browser/extract-audio', { path, root }),
|
||||||
|
|
||||||
|
|||||||
@@ -26,9 +26,8 @@ export {
|
|||||||
useChat,
|
useChat,
|
||||||
useSlashCommands,
|
useSlashCommands,
|
||||||
useAttachments,
|
useAttachments,
|
||||||
useAudioRecording,
|
|
||||||
} from './apps/Chat';
|
} from './apps/Chat';
|
||||||
export type { UseEmbeddableChatType, UseChatType, UseAttachmentsType, UseAudioRecordingType } from './apps/Chat';
|
export type { UseEmbeddableChatType, UseChatType, UseAttachmentsType } from './apps/Chat';
|
||||||
export * from './apps/Chat/types';
|
export * from './apps/Chat/types';
|
||||||
export { SessionList, ChatDetailPanel } from './apps/ChatHistory';
|
export { SessionList, ChatDetailPanel } from './apps/ChatHistory';
|
||||||
export type { SelectedSession } from './apps/ChatHistory';
|
export type { SelectedSession } from './apps/ChatHistory';
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Vorlesen",
|
"readAloud": "Vorlesen",
|
||||||
"ocr": "Text extrahieren (OCR)",
|
"ocr": "Text extrahieren (OCR)",
|
||||||
"transcribe": "Transkribieren",
|
|
||||||
"extractAudio": "Audio extrahieren",
|
"extractAudio": "Audio extrahieren",
|
||||||
"extract": "Extrahieren",
|
"extract": "Extrahieren",
|
||||||
"saveNextTo": "Neben dem Original speichern",
|
"saveNextTo": "Neben dem Original speichern",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Nach oben scrollen",
|
"scrollTop": "Nach oben scrollen",
|
||||||
"speechFailed": "Sprachaudio konnte nicht erzeugt werden",
|
"speechFailed": "Sprachaudio konnte nicht erzeugt werden",
|
||||||
"ocrFailed": "Text konnte nicht aus dem Bild extrahiert werden",
|
"ocrFailed": "Text konnte nicht aus dem Bild extrahiert werden",
|
||||||
"transcribeFailed": "Audio konnte nicht transkribiert werden",
|
|
||||||
"extractAudioFailed": "Audio konnte nicht aus dem Video extrahiert werden",
|
"extractAudioFailed": "Audio konnte nicht aus dem Video extrahiert werden",
|
||||||
"extractFailed": "Archiv konnte nicht extrahiert werden",
|
"extractFailed": "Archiv konnte nicht extrahiert werden",
|
||||||
"savedAs": "Gespeichert als “{{name}}”",
|
"savedAs": "Gespeichert als “{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Aus der Warteschlange entfernen",
|
"removeFromQueue": "Aus der Warteschlange entfernen",
|
||||||
"send": "Senden",
|
"send": "Senden",
|
||||||
"startHint": "Senden Sie eine Nachricht, um zu beginnen",
|
"startHint": "Senden Sie eine Nachricht, um zu beginnen",
|
||||||
"jumpToLatest": "Zur neuesten Nachricht springen",
|
"jumpToLatest": "Zur neuesten Nachricht springen"
|
||||||
"noAudio": "Es wurde kein Audio aufgenommen"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "Der Agent dieses Panels existiert nicht mehr.",
|
"gone": "Der Agent dieses Panels existiert nicht mehr.",
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Read Aloud",
|
"readAloud": "Read Aloud",
|
||||||
"ocr": "Extract Text (OCR)",
|
"ocr": "Extract Text (OCR)",
|
||||||
"transcribe": "Transcribe",
|
|
||||||
"extractAudio": "Extract Audio",
|
"extractAudio": "Extract Audio",
|
||||||
"extract": "Extract",
|
"extract": "Extract",
|
||||||
"saveNextTo": "Save next to original",
|
"saveNextTo": "Save next to original",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Scroll to top",
|
"scrollTop": "Scroll to top",
|
||||||
"speechFailed": "Failed to generate speech audio",
|
"speechFailed": "Failed to generate speech audio",
|
||||||
"ocrFailed": "Failed to extract text from image",
|
"ocrFailed": "Failed to extract text from image",
|
||||||
"transcribeFailed": "Failed to transcribe audio",
|
|
||||||
"extractAudioFailed": "Failed to extract audio from video",
|
"extractAudioFailed": "Failed to extract audio from video",
|
||||||
"extractFailed": "Failed to extract archive",
|
"extractFailed": "Failed to extract archive",
|
||||||
"savedAs": "Saved as “{{name}}”",
|
"savedAs": "Saved as “{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Remove from queue",
|
"removeFromQueue": "Remove from queue",
|
||||||
"send": "Send",
|
"send": "Send",
|
||||||
"startHint": "Send a message to start",
|
"startHint": "Send a message to start",
|
||||||
"jumpToLatest": "Jump to latest message",
|
"jumpToLatest": "Jump to latest message"
|
||||||
"noAudio": "No audio was captured"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "This panel’s agent no longer exists.",
|
"gone": "This panel’s agent no longer exists.",
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Leer en voz alta",
|
"readAloud": "Leer en voz alta",
|
||||||
"ocr": "Extraer texto (OCR)",
|
"ocr": "Extraer texto (OCR)",
|
||||||
"transcribe": "Transcribir",
|
|
||||||
"extractAudio": "Extraer audio",
|
"extractAudio": "Extraer audio",
|
||||||
"extract": "Extraer",
|
"extract": "Extraer",
|
||||||
"saveNextTo": "Guardar junto al original",
|
"saveNextTo": "Guardar junto al original",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Ir arriba",
|
"scrollTop": "Ir arriba",
|
||||||
"speechFailed": "No se pudo generar el audio de voz",
|
"speechFailed": "No se pudo generar el audio de voz",
|
||||||
"ocrFailed": "No se pudo extraer el texto de la imagen",
|
"ocrFailed": "No se pudo extraer el texto de la imagen",
|
||||||
"transcribeFailed": "No se pudo transcribir el audio",
|
|
||||||
"extractAudioFailed": "No se pudo extraer el audio del vídeo",
|
"extractAudioFailed": "No se pudo extraer el audio del vídeo",
|
||||||
"extractFailed": "No se pudo extraer el archivo comprimido",
|
"extractFailed": "No se pudo extraer el archivo comprimido",
|
||||||
"savedAs": "Guardado como “{{name}}”",
|
"savedAs": "Guardado como “{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Quitar de la cola",
|
"removeFromQueue": "Quitar de la cola",
|
||||||
"send": "Enviar",
|
"send": "Enviar",
|
||||||
"startHint": "Envía un mensaje para empezar",
|
"startHint": "Envía un mensaje para empezar",
|
||||||
"jumpToLatest": "Ir al último mensaje",
|
"jumpToLatest": "Ir al último mensaje"
|
||||||
"noAudio": "No se captó ningún audio"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "El agente de este panel ya no existe.",
|
"gone": "El agente de este panel ya no existe.",
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Lire à voix haute",
|
"readAloud": "Lire à voix haute",
|
||||||
"ocr": "Extraire le texte (OCR)",
|
"ocr": "Extraire le texte (OCR)",
|
||||||
"transcribe": "Transcrire",
|
|
||||||
"extractAudio": "Extraire l'audio",
|
"extractAudio": "Extraire l'audio",
|
||||||
"extract": "Extraire",
|
"extract": "Extraire",
|
||||||
"saveNextTo": "Enregistrer à côté de l'original",
|
"saveNextTo": "Enregistrer à côté de l'original",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Remonter en haut",
|
"scrollTop": "Remonter en haut",
|
||||||
"speechFailed": "Échec de la génération de l'audio vocal",
|
"speechFailed": "Échec de la génération de l'audio vocal",
|
||||||
"ocrFailed": "Échec de l'extraction du texte de l'image",
|
"ocrFailed": "Échec de l'extraction du texte de l'image",
|
||||||
"transcribeFailed": "Échec de la transcription de l'audio",
|
|
||||||
"extractAudioFailed": "Échec de l'extraction de l'audio de la vidéo",
|
"extractAudioFailed": "Échec de l'extraction de l'audio de la vidéo",
|
||||||
"extractFailed": "Échec de l'extraction de l'archive",
|
"extractFailed": "Échec de l'extraction de l'archive",
|
||||||
"savedAs": "Enregistré sous “{{name}}”",
|
"savedAs": "Enregistré sous “{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Retirer de la file",
|
"removeFromQueue": "Retirer de la file",
|
||||||
"send": "Envoyer",
|
"send": "Envoyer",
|
||||||
"startHint": "Envoyez un message pour commencer",
|
"startHint": "Envoyez un message pour commencer",
|
||||||
"jumpToLatest": "Aller au message le plus récent",
|
"jumpToLatest": "Aller au message le plus récent"
|
||||||
"noAudio": "Aucun audio n'a été capté"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "L'agent de ce panneau n'existe plus.",
|
"gone": "L'agent de ce panneau n'existe plus.",
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Ler em voz alta",
|
"readAloud": "Ler em voz alta",
|
||||||
"ocr": "Extrair texto (OCR)",
|
"ocr": "Extrair texto (OCR)",
|
||||||
"transcribe": "Transcrever",
|
|
||||||
"extractAudio": "Extrair áudio",
|
"extractAudio": "Extrair áudio",
|
||||||
"extract": "Extrair",
|
"extract": "Extrair",
|
||||||
"saveNextTo": "Guardar ao lado do original",
|
"saveNextTo": "Guardar ao lado do original",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Ir para o topo",
|
"scrollTop": "Ir para o topo",
|
||||||
"speechFailed": "Falha ao gerar o áudio de fala",
|
"speechFailed": "Falha ao gerar o áudio de fala",
|
||||||
"ocrFailed": "Falha ao extrair texto da imagem",
|
"ocrFailed": "Falha ao extrair texto da imagem",
|
||||||
"transcribeFailed": "Falha ao transcrever o áudio",
|
|
||||||
"extractAudioFailed": "Falha ao extrair o áudio do vídeo",
|
"extractAudioFailed": "Falha ao extrair o áudio do vídeo",
|
||||||
"extractFailed": "Falha ao extrair o arquivo",
|
"extractFailed": "Falha ao extrair o arquivo",
|
||||||
"savedAs": "Guardado como “{{name}}”",
|
"savedAs": "Guardado como “{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Remover da fila",
|
"removeFromQueue": "Remover da fila",
|
||||||
"send": "Enviar",
|
"send": "Enviar",
|
||||||
"startHint": "Envie uma mensagem para começar",
|
"startHint": "Envie uma mensagem para começar",
|
||||||
"jumpToLatest": "Ir para a mensagem mais recente",
|
"jumpToLatest": "Ir para a mensagem mais recente"
|
||||||
"noAudio": "Não foi captado áudio"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "O agente deste painel já não existe.",
|
"gone": "O agente deste painel já não existe.",
|
||||||
|
|||||||
@@ -152,7 +152,6 @@
|
|||||||
"fileViewer": {
|
"fileViewer": {
|
||||||
"readAloud": "Läs upp",
|
"readAloud": "Läs upp",
|
||||||
"ocr": "Extrahera text (OCR)",
|
"ocr": "Extrahera text (OCR)",
|
||||||
"transcribe": "Transkribera",
|
|
||||||
"extractAudio": "Extrahera ljud",
|
"extractAudio": "Extrahera ljud",
|
||||||
"extract": "Extrahera",
|
"extract": "Extrahera",
|
||||||
"saveNextTo": "Spara bredvid originalet",
|
"saveNextTo": "Spara bredvid originalet",
|
||||||
@@ -169,7 +168,6 @@
|
|||||||
"scrollTop": "Rulla till toppen",
|
"scrollTop": "Rulla till toppen",
|
||||||
"speechFailed": "Det gick inte att generera talljud",
|
"speechFailed": "Det gick inte att generera talljud",
|
||||||
"ocrFailed": "Det gick inte att extrahera text från bilden",
|
"ocrFailed": "Det gick inte att extrahera text från bilden",
|
||||||
"transcribeFailed": "Det gick inte att transkribera ljudet",
|
|
||||||
"extractAudioFailed": "Det gick inte att extrahera ljud från videon",
|
"extractAudioFailed": "Det gick inte att extrahera ljud från videon",
|
||||||
"extractFailed": "Det gick inte att extrahera arkivet",
|
"extractFailed": "Det gick inte att extrahera arkivet",
|
||||||
"savedAs": "Sparad som ”{{name}}”",
|
"savedAs": "Sparad som ”{{name}}”",
|
||||||
@@ -404,8 +402,7 @@
|
|||||||
"removeFromQueue": "Ta bort från kön",
|
"removeFromQueue": "Ta bort från kön",
|
||||||
"send": "Skicka",
|
"send": "Skicka",
|
||||||
"startHint": "Skicka ett meddelande för att börja",
|
"startHint": "Skicka ett meddelande för att börja",
|
||||||
"jumpToLatest": "Hoppa till senaste meddelandet",
|
"jumpToLatest": "Hoppa till senaste meddelandet"
|
||||||
"noAudio": "Inget ljud spelades in"
|
|
||||||
},
|
},
|
||||||
"agent": {
|
"agent": {
|
||||||
"gone": "Den här panelens agent finns inte längre.",
|
"gone": "Den här panelens agent finns inte längre.",
|
||||||
|
|||||||
Reference in New Issue
Block a user