// Whisper transcription, always in the audio's own language. // // Detect the language first, then transcribe in it. Whisper's translate mode is English-only, so it // cannot honour "translate into " for anything else — rather than silently returning a // rough English rendering of Portuguese audio, transcription is always faithful to the source. // // The detection must win when Whisper is confident. `spokenLanguages` only breaks ties on clips // Whisper is genuinely unsure about, where its low-confidence guess defaults to English. // // The transcription pass is tuned for one person talking into a phone, which is what this is mostly // used for. See the constants below: the defaults whisper.cpp ships are aimed at clean dictation and // produce loops, clipped words and breath-length paragraph breaks on real voice memos. // // Both requests pass `timeout: false`. Bun's fetch aborts at 300s by default, and whisper.cpp runs at // roughly 8x realtime, so anything over ~40 minutes of audio died with "The operation timed out" // about 20 seconds short of the answer. AbortSignal.timeout() does not raise that ceiling — only // disabling the built-in timeout does. // Bun's fetch accepts `timeout: false` to disable its built-in 300s ceiling. The DOM RequestInit // type this project compiles against does not declare it, so widen it here rather than casting at // each call site. type BunRequestInit = RequestInit & { timeout?: boolean | number }; type TranscribeArgs = { file: File | Blob; whisperUrl: string; spokenLanguages: string[]; }; export type TranscribeResult = { text: string; detectedLanguage: string; /** Raw Whisper segments, kept for speaker attribution (conversation.ts). Not part of the HTTP * response — index.ts decides what leaves the sidecar. */ segments: WhisperSegment[]; }; // Above this probability Whisper's own detection is taken as authoritative. Below it the clip is // treated as ambiguous and the user's spoken languages break the tie. const CONFIDENT_DETECTION = 0.5; // whisper.cpp's "language" field uses full names ("english"); ISO codes live in // `language_probabilities`. Map covers what the user can pick in Profile settings. const LANG_NAME_TO_ISO: Record = { english: 'en', portuguese: 'pt', spanish: 'es', french: 'fr', german: 'de', italian: 'it', dutch: 'nl', russian: 'ru', chinese: 'zh', japanese: 'ja', korean: 'ko', }; // Whisper reuses its own decoded output as the prompt for the next 30-second window, which is what // makes it fall into loops ("I know. I know. I know.") once it slips. `max_context` is the budget for // that carried text, and setting it to 0 stops the loops — but the same switch also disables the // initial prompt, since whisper.cpp gates both on `n_max_text_ctx > 0`. // // The way to keep the prompt and still starve the loop: `carry_initial_prompt` puts the prompt in a // static slot that is filled first and in full, and only the leftover budget goes to decoded text. // Every prompt below is comfortably longer than this budget, so there is never any leftover — the // vocabulary and style reach every window and none of the model's own output does. const PROMPT_CONTEXT_TOKENS = 96; // Whisper imitates whatever it is primed with; it does not take instructions. So each prompt is a // worked example of the output wanted — one person talking, punctuated the way it should come back — // and nothing else. An earlier version opened by *describing* the format ("Commas separate clauses, // and apostrophes are written properly") and that backfired: priming words which never occur in // speech put them in play, and a character surname came back from a Monk episode as "Commas". // Everything here is ordinary conversational language for that reason. // // Each is written in its own language, since an English prompt on Portuguese audio biases the // output, and each deliberately runs past PROMPT_CONTEXT_TOKENS — see above for why the length is // load-bearing. Overflow is trimmed from the front, so nothing depends on the opening words. // // Languages without an entry fall through to no prompt at all, which is handled explicitly below. // // Every prompt uses its language's full diacritic set (and, for a non-Latin script, its own // punctuation — Japanese 。/、, Greek ; as a question mark, Arabic ، and ؟) in ordinary words, never // forced in. Omitting one, e.g. writing Swedish without å ä ö, teaches Whisper to drop it too. const TRANSCRIPT_PROMPTS: Record = { en: 'Hey, sorry for the late reply — I was stuck in a meeting all morning and only just got out. ' + "I had a look at the document you sent over, and honestly, I think it's mostly fine. There are " + "a couple of things I'd change in the second half, but nothing serious. The introduction reads " + 'really well. Let me know when you have a few minutes and we can talk it through properly. ' + "I'm around all afternoon, so whenever suits you is fine by me. Thanks again for putting it " + 'together, I know it took a while.', pt: 'Olá, desculpa a demora a responder — estive numa reunião toda a manhã e só agora consegui ' + 'sair. Já vi o documento que enviaste e, sinceramente, acho que está quase tudo bem. Há duas ' + 'ou três coisas que mudava na segunda metade, mas nada de grave. A introdução está muito bem ' + 'escrita. Diz-me quando tiveres uns minutos e falamos com calma. Estou por aqui toda a tarde, ' + 'por isso quando te der jeito está bem para mim. Obrigado outra vez por teres tratado disto, ' + 'sei que deu trabalho.', es: 'Hola, perdona la tardanza en responder — he estado toda la mañana en una reunión y recién ahora ' + 'he podido salir. Ya vi el documento que me enviaste y, la verdad, creo que está bastante bien. ' + 'Hay un par de cosas que cambiaría en la segunda mitad, pero nada grave. La introducción está muy ' + 'bien escrita. Avísame cuando tengas unos minutos y lo hablamos con calma. Estoy disponible toda ' + 'la tarde, así que cuando te venga bien está perfecto para mí. Gracias otra vez por armarlo, sé ' + 'que llevó tiempo.', de: 'Hallo, entschuldige die späte Antwort — ich saß den ganzen Vormittag in einer Besprechung und ' + 'bin gerade erst rausgekommen. Ich habe mir das Dokument angeschaut, das du geschickt hast, und ' + 'ehrlich gesagt finde ich es größtenteils gut. In der zweiten Hälfte würde ich ein paar ' + 'Kleinigkeiten ändern, aber nichts Ernstes. Die Einleitung liest sich wirklich gut. Sag mir ' + 'Bescheid, wenn du später ein paar Minuten hast, dann können wir in Ruhe darüber reden. Ich bin ' + 'den ganzen Nachmittag über da, also wann immer es dir passt, ist mir recht. Danke nochmal, dass ' + 'du dir die Mühe gemacht hast, ich weiß, das hat Zeit gekostet.', fr: "Salut, désolé pour la réponse tardive — j'étais coincé en réunion toute la matinée et je viens " + "tout juste d'en sortir. J'ai regardé le document que tu as envoyé, et honnêtement, je trouve que " + "c'est plutôt bien. Il y a deux ou trois choses que je changerais dans la deuxième moitié, mais " + "rien de grave. L'introduction est vraiment bien écrite. Dis-moi quand tu as quelques minutes et " + "on en parle tranquillement. Je suis disponible tout l'après-midi, donc quand ça t'arrange, ça me " + "va. Merci encore de l'avoir préparé, je sais que ça a pris du temps.", sv: 'Hej, förlåt att jag inte svarat tidigare — jag satt i möte hela förmiddagen och kom precis loss. ' + 'Jag har tittat på dokumentet du skickade, och ärligt talat tycker jag att det mesta är bra. Det ' + 'är några saker jag skulle ändra i andra halvan, men inget allvarligt. Inledningen är verkligen ' + 'bra skriven. Säg till när du har några minuter så pratar vi igenom det ordentligt. Jag är ledig ' + 'hela eftermiddagen, så när det passar dig passar det mig. Tack igen för att du satte ihop det, ' + 'jag vet att det tog ett tag.', it: 'Ciao, scusa il ritardo nella risposta — sono stato in riunione tutta la mattina e sono appena ' + "uscito. Ho dato un'occhiata al documento che mi hai mandato e, sinceramente, penso che vada bene " + 'quasi tutto. Ci sono un paio di cose che cambierei nella seconda metà, ma niente di grave. ' + "L'introduzione è scritta davvero bene. Fammi sapere quando hai qualche minuto così ne parliamo " + 'con calma. Sono libero tutto il pomeriggio, quindi va bene quando ti fa comodo. Grazie ancora ' + 'per averlo messo insieme, so che ci è voluto tempo.', nl: 'Hoi, sorry voor het late antwoord — ik zat de hele ochtend in een vergadering en ben er net pas ' + 'uit. Ik heb naar het document gekeken dat je hebt gestuurd, en eerlijk gezegd vind ik het ' + 'grotendeels goed. Er zijn een paar dingen die ik in de tweede helft zou veranderen, maar niets ' + 'ergs. De inleiding leest heel prettig. Laat het me weten wanneer je een paar minuten hebt, dan ' + 'bespreken we het rustig. Ik ben de hele middag beschikbaar, dus wanneer het jou uitkomt is voor ' + 'mij prima. Bedankt nogmaals dat je het hebt samengesteld, ik weet dat het tijd heeft gekost.', pl: 'Cześć, przepraszam za późną odpowiedź — siedziałem cały ranek na spotkaniu i w końcu udało mi ' + 'się z niego wyrwać. Zerknąłem na dokument, który przesłałeś, i szczerze mówiąc, myślę, że ' + 'większość jest w porządku. Jest kilka rzeczy, które zmieniłbym w drugiej połowie, ale nic ' + 'poważnego. Wstęp jest naprawdę dobrze napisany. Daj znać, kiedy będziesz mieć chwilę, możemy to ' + 'spokojnie omówić. Jestem wolny całe popołudnie, więc kiedy ci pasuje, mi też pasuje. Dzięki ' + 'jeszcze raz za to, że to zebrałeś, wiem, że zajęło to trochę czasu.', da: 'Hej, undskyld det sene svar — jeg sad i møde hele formiddagen og er lige kommet ud. Jeg har ' + 'kigget på dokumentet, du sendte, og ærligt talt synes jeg det meste er fint. Der er et par ting, ' + 'jeg ville ændre i anden halvdel, men ikke noget alvorligt. Indledningen er skrevet rigtig godt. ' + 'Sig til, når du har et par minutter, så snakker vi ordentligt om det. Jeg er ledig hele ' + 'eftermiddagen, så det passer mig, når det passer dig. Tak igen fordi du satte det sammen, jeg ' + 'ved det tog tid.', no: 'Hei, beklager det sene svaret — jeg satt i møte hele formiddagen og er akkurat ferdig. Jeg har ' + 'sett på dokumentet du sendte, og ærlig talt synes jeg det meste er bra. Det er et par ting jeg ' + 'ville endret i andre halvdel, men ingenting alvorlig. Innledningen er skrevet veldig bra. Si ' + 'ifra når du har noen minutter, så tar vi en skikkelig prat om det. Jeg er ledig hele ' + 'ettermiddagen, så det som passer deg passer meg. Takk igjen for at du satte det sammen, jeg vet ' + 'det tok tid.', fi: 'Hei, anteeksi myöhäinen vastaus — istuin koko aamupäivän palaverissa ja pääsin juuri äsken pois. ' + 'Katsoin dokumentin, jonka lähetit, ja rehellisesti sanottuna suurin osa siitä on ihan hyvä. ' + 'Loppupuoliskolla on pari asiaa, jotka muuttaisin, mutta ei mitään vakavaa. Johdanto on ' + 'kirjoitettu tosi hyvin. Sano, kun sinulla on hetki aikaa, niin jutellaan siitä kunnolla. Olen ' + 'vapaana koko iltapäivän, joten mikä tahansa sopii sinulle, sopii minulle. Kiitos vielä kerran, ' + 'kun kokosit sen, tiedän että siihen meni aikaa.', tr: 'Selam, geç cevap verdiğim için kusura bakma — bütün sabah toplantıdaydım ve az önce çıkabildim. ' + 'Gönderdiğin dokümana baktım ve açıkçası çoğunlukla iyi olduğunu düşünüyorum. İkinci yarıda ' + 'değiştireceğim birkaç şey var ama ciddi bir şey değil. Giriş kısmı gerçekten iyi yazılmış. ' + 'Birkaç dakikan olduğunda haber ver, düzgünce konuşuruz. Bütün öğleden sonra müsaitim, yani sana ' + 'uyan zaman bana da uyar. Bunu hazırladığın için tekrar teşekkürler, biliyorum zaman almıştır.', cs: 'Ahoj, promiň za pozdní odpověď — celé dopoledne jsem byl na poradě a teprve teď jsem se dostal ' + 'ven. Podíval jsem se na dokument, cos poslal, a upřímně řečeno, myslím, že většina je v pořádku. ' + 'V druhé polovině bych pár věcí změnil, ale nic vážného. Úvod je napsaný fakt dobře. Dej vědět, ' + 'až budeš mít pár minut, můžeme to pořádně probrat. Celé odpoledne mám volno, takže kdy se ti to ' + 'bude hodit, mně to vyhovuje. Díky ještě jednou, žes to dal dohromady, vím, že to zabralo čas.', el: 'Γεια σου, συγγνώμη για την καθυστερημένη απάντηση — ήμουν σε συνάντηση όλο το πρωί και μόλις ' + 'τώρα βγήκα. Είδα το έγγραφο που έστειλες και, ειλικρινά, νομίζω πως είναι σχεδόν όλο εντάξει. ' + 'Υπάρχουν δύο τρία πράγματα που θα άλλαζα στο δεύτερο μισό, αλλά τίποτα σοβαρό. Η εισαγωγή είναι ' + 'γραμμένη πολύ καλά. Έχεις λίγα λεπτά να τα πούμε με την ησυχία μας; Είμαι ελεύθερος όλο το ' + 'απόγευμα, οπότε όποτε σε βολεύει εμένα με βολεύει. Ευχαριστώ ξανά που το ετοίμασες, ξέρω πως ' + 'πήρε χρόνο.', uk: 'Привіт, вибач за пізню відповідь — я весь ранок сидів на нараді і щойно звідти вийшов. Подивився ' + 'документ, який ти надіслав, і чесно кажучи, мої думки такі — майже все добре. У мене є пара ' + 'речей, які я б змінив у другій половині, але нічого серйозного. Вступ написаний дуже добре. ' + 'Напиши, коли матимеш кілька хвилин, і спокійно все обговоримо. Я вільний увесь день, тобто коли тобі зручно, ' + 'мені теж підходить. Дякую ще раз, що зібрав це докупи, знаю, що це зайняло час.', ru: 'Привет, извини, что долго не отвечал — всё утро просидел на встрече и только что освободился. ' + 'Посмотрел документ, который ты прислал, и, честно говоря, думаю, что почти всё нормально. Есть ' + 'пара моментов, которые я бы изменил во второй половине, но ничего серьёзного. Введение написано ' + 'правда хорошо. Напиши, когда будет пара свободных минут, спокойно всё обсудим. Я свободен весь ' + 'день, так что когда тебе удобно, мне тоже подходит. Спасибо ещё раз, что всё это собрал, знаю, ' + 'что это заняло время.', ro: 'Salut, scuze pentru răspunsul întârziat — am fost toată dimineața într-o ședință și abia acum am ' + 'scăpat. M-am uitat pe documentul pe care mi l-ai trimis și, sincer, cred că e aproape totul ' + 'bine. Sunt vreo două-trei lucruri pe care le-aș schimba în a doua jumătate, dar nimic grav. ' + 'Introducerea e scrisă chiar bine. Spune-mi când ai câteva minute și vorbim pe îndelete despre ' + 'asta. Sunt liber toată după-amiaza, deci oricând îți convine mie îmi convine. Mulțumesc încă o ' + 'dată că ai pus totul cap la cap, știu că a durat.', hu: 'Szia, bocs a késői válaszért — egész délelőtt gyűlésen ültem, és csak most sikerült ' + 'kiszabadulnom. Megnéztem a dokumentumot, amit küldtél, és őszintén szólva szerintem szinte ' + 'minden rendben van. Van pár dolog, amit megváltoztatnék a második felében, de semmi komoly. A ' + 'bevezető nagyon jól van megírva. Szólj, ha van pár szabad perced, és nyugodtan átbeszéljük. ' + 'Egész délután érek rá, szóval amikor neked jó, nekem is megfelel. Köszönöm még egyszer, hogy ' + 'összeraktad, tudom, hogy hosszú időbe telt.', ja: '返信が遅くなってごめん。朝からずっと会議で、今やっと抜けられたところ。送ってもらった資料、見たよ。' + '正直、ほとんど問題ないと思う。後半のところで直したい部分が二つ三つあるけど、大したことじゃない。' + '導入の部分はすごくよく書けてると思う。時間があるときに教えて、ゆっくり話そう。' + '午後はずっと空いてるから、都合のいいときでいいよ。まとめてくれてありがとう、時間かかったよね。', zh: '抱歉这么晚才回你——我一早上都在开会,刚出来。你发的那份文件我看过了,说实话,大部分都挺好的。' + '后半部分有几个地方我想改一下,但都不是什么大问题。开头写得真的很不错。你什么时候有空跟我说一声,我们好好聊聊。' + '我整个下午都有空,所以你什么时候方便都行。再次谢谢你把这个整理出来,我知道挺花时间的。', ko: '답장이 늦어서 미안해. 오전 내내 회의하다가 방금 겨우 빠져나왔어. 보내준 문서 봤는데, 솔직히 거의 다 괜찮은 것 같아. 후반부에 고치고 싶은 부분이 두어 개 있긴 한데, ' + '심각한 건 아니야. 서론은 진짜 잘 쓴 것 같아. 시간 될 때 얘기해줘, 그럼 차분히 얘기해보자. 오후 내내 시간 있으니까 편한 때 아무 때나 괜찮아. 정리해줘서 다시 한번 ' + '고마워, 시간 좀 걸렸을 텐데.', ar: 'مرحبًا، آسف على تأخري في الرد — كنت في اجتماع طوال الصباح ولتوّي خرجت منه. اطّلعت على المستند ' + 'الذي أرسلته، وبصراحة أعتقد أن معظمه جيد. هناك بعض الأشياء التي أود تغييرها في النصف الثاني، لكن ' + 'لا شيء خطير. المقدمة مكتوبة بشكل جيد جدًا. هل يمكن أن تخبرني متى تكون لديك بضع دقائق لنتحدث ' + 'بهدوء؟ أنا متفرغ طوال بعد الظهر، فأي وقت يناسبك يناسبني. شكرًا مرة أخرى على إعدادك لهذا، أعرف أن ' + 'الأمر استغرق وقتًا.', he: 'היי, סליחה על התגובה המאוחרת — הייתי בישיבה כל הבוקר ורק עכשיו יצאתי. הסתכלתי על המסמך ששלחת, ' + 'ובכנות, אני חושב שרוב הדברים בסדר. יש כמה דברים שהייתי משנה בחצי השני, אבל שום דבר רציני. הפתיחה ' + 'כתובה ממש טוב. תגיד לי מתי יש לך כמה דקות ונדבר על זה בנחת. אני פנוי כל אחר הצהריים, אז מתי שנוח ' + 'לך נוח גם לי. תודה שוב שסידרת את זה, אני יודע שזה לקח זמן.', hi: 'हाय, देर से जवाब देने के लिए माफ़ करना — मैं पूरी सुबह मीटिंग में था और अभी-अभी निकल पाया। तुमने ' + 'जो डॉक्यूमेंट भेजा था, वो देख लिया, और सच कहूं तो ज़्यादातर हिस्सा ठीक लग रहा है। दूसरे हिस्से ' + 'में कुछ चीज़ें हैं जो मैं बदलना चाहूंगा, पर कुछ गंभीर नहीं है। शुरुआत बहुत अच्छी लिखी गई है। जब ' + 'भी कुछ मिनट मिलें बता देना, फिर आराम से बात करते हैं। पूरी दोपहर मैं फ्री हूं, तो जब भी तुम्हें ' + 'सही लगे मुझे ठीक है। इसे तैयार करने के लिए फिर से शुक्रिया, पता है इसमें वक़्त लगा होगा।', }; // Beam search rather than the default greedy decode (whisper.cpp ships `beam_size = -1`). It is the // single biggest quality lever here and it costs time, which this flow has — nothing waits on it. const BEAM_SIZE = 5; const BEST_OF = 5; // The whisper container already runs silero VAD, but its defaults assume clean dictation. Widen the // segments so they break at real pauses instead of at breaths, and pad the edges so word onsets are // not clipped — clipped onsets are what shows up in the output as a word split down the middle. const VAD_TUNING: Record = { vad_speech_pad_ms: '200', // default 30 vad_min_silence_duration_ms: '500', // default 100 — a breath is not a sentence boundary vad_samples_overlap: '0.2', // default 0.1 vad_max_speech_duration_s: '25', // default unbounded — keep a segment inside one 30s window }; // A silence this long reads as a new thought. Whisper reports segment times against the original // audio even when VAD is on (whisper.cpp maps them back through its VAD table), so these really are // gaps in the recording rather than artefacts of the filtering. const PARAGRAPH_GAP_SECONDS = 1.2; const MAX_PARAGRAPH_CHARS = 900; // OpenAI Whisper's own silence rule: a segment the model thinks is probably not speech *and* decodes // with low confidence is a hallucination rather than a transcript. whisper.cpp applies a subtly // different form of this internally, so it is worth re-applying to the values it reports — this is // what removes the stray words that appear over music and room tone. const NO_SPEECH_PROB = 0.6; const LOW_CONFIDENCE_LOGPROB = -1.0; export type WhisperSegment = { text?: string; start?: number; end?: number; no_speech_prob?: number; avg_logprob?: number; }; // Defaults are chosen so a segment missing either field is kept: dropping real speech is much worse // than keeping a stray word. export const isHallucinatedSilence = (segment: WhisperSegment) => (segment.no_speech_prob ?? 0) > NO_SPEECH_PROB && (segment.avg_logprob ?? 0) < LOW_CONFIDENCE_LOGPROB; // Whisper emits each segment with a leading space and now and then strands trailing punctuation in a // segment of its own, which is why the old `response_format=text` output contained lone "." lines. export const appendSegment = (paragraph: string, text: string) => { if (!paragraph) return text; return /^[.,;:!?…)\]]/.test(text) ? paragraph + text : `${paragraph} ${text}`; }; // Whisper returns one segment per utterance, not per paragraph. Rejoin them into prose and start a // new paragraph on a long pause, so a voice memo reads as something written rather than as a list of // fragments. export function toParagraphs(segments: WhisperSegment[]): string { const paragraphs: string[] = []; let current = ''; let previousEnd: number | null = null; for (const segment of segments) { const text = (segment.text ?? '').replace(/\s+/g, ' ').trim(); if (!text || isHallucinatedSilence(segment)) continue; const gap = previousEnd !== null && segment.start !== undefined ? segment.start - previousEnd : 0; // Only break an over-long paragraph where a sentence actually ended, never mid-thought. const overlong = current.length >= MAX_PARAGRAPH_CHARS && /[.!?…]["')\]]?$/.test(current); if (current && (gap >= PARAGRAPH_GAP_SECONDS || overlong)) { paragraphs.push(current); current = ''; } current = appendSegment(current, text); if (segment.end !== undefined) previousEnd = segment.end; } if (current) paragraphs.push(current); return paragraphs.join('\n\n'); } export async function transcribeAudio({ file, whisperUrl, spokenLanguages, }: TranscribeArgs): Promise { const base = whisperUrl.replace(/\/+$/, ''); // Step 1: detect language, and only fall back to the spoken-language bias when Whisper is unsure. const detectForm = new FormData(); detectForm.append('file', file); detectForm.append('temperature', '0.0'); detectForm.append('response_format', 'verbose_json'); detectForm.append('detect_language', 'true'); const detectRes = await fetch(`${base}/inference`, { method: 'POST', body: detectForm, timeout: false, } as BunRequestInit); if (!detectRes.ok) throw new Error(`Language detection failed (${detectRes.status})`); const detectJson = (await detectRes.json()) as { language?: string; detected_language?: string; detected_language_probability?: number; language_probabilities?: Record; }; const probs = detectJson.language_probabilities; const ranked = probs ? Object.entries(probs).sort((a, b) => b[1] - a[1]) : []; const [topCode, topProb] = ranked[0] ?? []; let detectedLanguage: string; if (topCode && (topProb ?? 0) >= CONFIDENT_DETECTION) { // Whisper is sure. Trust it even when it is not a language the user speaks — that is exactly the // case translation exists for. detectedLanguage = topCode; } else if (ranked.length > 0 && spokenLanguages.length > 0) { // Genuinely ambiguous (usually a short clip). Break the tie toward a language the user speaks, // since Whisper's low-confidence guess defaults to English regardless of the audio. const bestSpoken = spokenLanguages .map((code) => [code, probs?.[code] ?? 0] as const) .sort((a, b) => b[1] - a[1])[0]; detectedLanguage = bestSpoken?.[0] ?? topCode ?? 'en'; } else if (topCode) { detectedLanguage = topCode; } else { // No probabilities field (non-whisper.cpp backend) — try to map the reported name. const raw = (detectJson.detected_language ?? detectJson.language ?? 'en').toLowerCase(); detectedLanguage = LANG_NAME_TO_ISO[raw] ?? raw; } // Step 2: transcribe in the detected language const transcribeForm = new FormData(); transcribeForm.append('file', file); transcribeForm.append('language', detectedLanguage); transcribeForm.append('temperature', '0.0'); transcribeForm.append('temperature_inc', '0.2'); transcribeForm.append('beam_size', String(BEAM_SIZE)); transcribeForm.append('best_of', String(BEST_OF)); transcribeForm.append('suppress_nst', 'true'); transcribeForm.append('response_format', 'verbose_json'); // verbose_json re-runs a full language auto-detect unless this is set. Step 1 already answered // that question, so it is pure duplicated work on every transcription. transcribeForm.append('no_language_probabilities', 'true'); // Segment timestamps drive the paragraphing below and are kept; per-token timestamps are extra // work on every segment and nothing here reads them. transcribeForm.append('token_timestamps', 'false'); const prompt = TRANSCRIPT_PROMPTS[detectedLanguage]; if (prompt) { transcribeForm.append('prompt', prompt); transcribeForm.append('carry_initial_prompt', 'true'); transcribeForm.append('max_context', String(PROMPT_CONTEXT_TOKENS)); } else { // Nothing worth protecting the budget for, so close the loop-feeding path outright. transcribeForm.append('max_context', '0'); } for (const [key, value] of Object.entries(VAD_TUNING)) transcribeForm.append(key, value); const res = await fetch(`${base}/inference`, { method: 'POST', body: transcribeForm, timeout: false, } as BunRequestInit); if (!res.ok) throw new Error(`Transcription failed (${res.status})`); const json = (await res.json()) as { text?: string; segments?: WhisperSegment[] }; // Fall back to the flat text for any backend that does not return segments. const segments = json.segments ?? []; const text = segments.length > 0 ? toParagraphs(segments) : (json.text ?? '').trim(); return { text, detectedLanguage, segments }; }