Streaming Speech to Text
Send audio over a WebSocket and get transcripts back as each turn ends — the transcription half of the realtime socket, without the agent. It is built for voice-agent frameworks that run their own turn detection and their own LLM: you decide when a turn is over, and you can let the session follow a caller who switches language.
Connection & auth
Same socket as realtime, with intent=transcription. Authenticate with your API key in an Authorization header (or ?api_key= where a header cannot be set). The key needs the naad product.
wss://platform.oogam.ai/v1/realtime?intent=transcription&language=hi
Authorization: Bearer sk-setu-…| Query parameter | Required | Description |
|---|---|---|
intent | yes | transcription. Without it you get the speech-to-speech agent. |
language | yes | The spoken language, as a short code: hi, gu, ta, en-IN … (the Live talk column in language codes). The model never auto-detects, so there is no default. |
auto_switch | no | true to follow a caller who changes language — see below. |
candidates | no | With auto_switch: comma-separated languages the session may switch to, e.g. hi,en-IN. Omit to allow any. |
min_confidence | no | With auto_switch: how sure a language hint must be (0–1) before switching. Default 0.6. |
vad_end_silence_ms | no | Reserved: accepted (700–5000) and passed to the engine, but the current engine does not change its end-of-turn timing for it. |
The session is ready when you receive session.ready. Audio sent before that is held and transcribed, not dropped.
Audio format
Send binary frames of 16-bit little-endian PCM, 16 kHz, mono — about 20 ms (640 bytes) per frame works well. Send continuously, silence included, as a microphone would. You only ever receive JSON text frames; a transcription session never sends audio.
Ending a turn: finalize
On its own, the engine closes a turn after about 0.7–0.8 s of silence and then transcribes it. Measured on the live engine, the transcript arrives about 0.8 s after the caller stops speaking; pauses of 0.7 s or less stay inside one turn.
If your application already knows the turn is over — a VAD, a semantic turn model such as Pipecat Smart Turn, a push-to-talk button — send {"type":"finalize"} as well. It tells the engine the turn has ended rather than leaving it to judge, so you do not wait out its silence window: measured on the live engine, the transcript arrives about 0.33 s after the end of speech instead of about 0.8 s. That transcript is marked from_finalize: true, and you get a finalize.ack at once. Turn detectors that wait for a final transcript (Pipecat's does) use that mark to stop waiting.
- Send
finalizeonce per turn, when your detector fires. A second one within 250 ms is acknowledged withskipped: trueand does nothing. - Keep streaming audio after a
finalize— the next turn starts with the next speech. - If a caller pauses for 0.8 s or more mid-thought, the engine closes the turn there and the rest arrives as a second transcript. Join transcripts until your own turn detector says the turn is over — Pipecat's user aggregator does this for you, so Smart Turn still decides when the agent replies.
- A transcript with
replace: truesupersedes the previous one rather than following it. The current engine does not send these, but handle the flag so your client keeps working when it does.
Changing language mid-call
Switch at any time with set_lang. To have the session switch by itself, turn on auto_switch — at connect with the query parameters above, or with a frame:
{"type":"set_lang","value":"hi","auto_switch":true,"candidates":["hi","en-IN"]}The engine sends a lang_hint when it hears a different language. With auto_switch on, a confident hint for one of your candidates re-pins the session and you receive language_changed with reason: "auto". After a switch the session will not switch again for 4 seconds, so a caller mixing Hindi and English does not flip it back and forth.
What to expect: the hint is about a turn that has already been transcribed, so the first sentence in the new language is transcribed in the old one — usually wrongly — and the switch applies from the next turn. The engine takes a few seconds to change language, so a sentence spoken while it switches can come back several seconds late. If the new language has no free capacity the switch is refused with language_switch_failed and the call carries on in the language it was in.
The engine notices some switches and not others. On live calls (15 November 2026), with the two languages listed as candidates:
| Session starts in | Caller switches to | Session follows |
|---|---|---|
| Hindi | English, Tamil, Bengali | From the second sentence |
| Hindi | Marathi | From the third sentence |
| Tamil | English | From the second sentence |
| Tamil | Hindi | From the third sentence |
| Hindi | Gujarati | Not yet |
| Gujarati | Hindi, English | Not yet |
| English | Hindi | Not yet |
For a pair that is not followed yet, start the session in the caller's language, or switch it yourself with set_lang — that applies at once. Test the mix your own callers use.
Message protocol
You send
| Message | Purpose |
|---|---|
| binary | Audio — 16 kHz mono PCM16. |
{"type":"finalize"} | The current turn is over; transcribe it now. |
{"type":"set_lang","value":"gu"} | Switch language. Add auto_switch, candidates, min_confidence to change automatic switching; auto_switch: false turns it off. |
{"type":"session.update","vad_end_silence_ms":1500} | Reserved (700–5000); the current engine does not act on it. |
{"type":"reset"} | Clear the engine's state without reconnecting. |
{"type":"keepalive"} | Ignored; for clients that need to send something. |
Agent frames — say, set_context, set_voice, set_tools — are refused with unsupported_in_transcription.
You receive
| Message | Meaning |
|---|---|
session.ready | Listening. Carries language, the audio format, vad_end_silence_ms, finalize.silence_ms and the auto_switch settings in force. |
transcript | text, language, is_final (always true), replace (supersedes the previous transcript) and, after a finalize, from_finalize: true. |
finalize.ack | Your finalize was received (skipped: true if it came within 250 ms of the last). |
lang_hint | detected language and confidence (0–1) — sent whether or not auto-switch is on. |
language_changed | from, to, reason (auto or client). |
language_switch_failed | An automatic switch was refused; language says where the call still is. |
session.update_ack | The new vad_end_silence_ms is in force. |
error | code and message. |
Pipecat
pipecat-oogam wraps this endpoint, our streaming text to speech and Sutra as Pipecat services. Its STT sends finalize whenever Pipecat's VAD reports that the user stopped speaking, so Smart Turn — Pipecat's default turn detector — decides when a turn is over, with no wait for the engine's own end-of-turn silence. A complete agent, install to first words, is on the Pipecat integration page.
pip install pipecat-oogam
from pipecat.transcriptions.language import Language
from pipecat_oogam import OogamSTTService, OogamTTSService, OogamLLMService
stt = OogamSTTService(
api_key=os.environ["OOGAM_API_KEY"],
settings=OogamSTTService.Settings(
language=Language.HI,
auto_switch=True,
candidates=[Language.HI, Language.EN_IN],
),
)Python example
import asyncio, json, os
import websockets # pip install websockets
URL = "wss://platform.oogam.ai/v1/realtime?intent=transcription&language=hi"
async def main(pcm_frames):
headers = {"Authorization": f"Bearer {os.environ['OOGAM_API_KEY']}"}
async with websockets.connect(URL, additional_headers=headers) as ws:
ready = json.loads(await ws.recv()) # {"type": "session.ready", ...}
print("listening in", ready["language"])
async def send_audio():
for frame in pcm_frames: # 16 kHz mono PCM16, ~20 ms each
await ws.send(frame)
await asyncio.sleep(0.02)
await ws.send(json.dumps({"type": "finalize"})) # your VAD says: done
async def receive():
async for raw in ws:
msg = json.loads(raw)
if msg["type"] == "transcript":
print(msg["language"], msg["text"])
elif msg["type"] == "error":
print("error:", msg["code"], msg["message"])
await asyncio.gather(send_audio(), receive())Errors
Refused at connect, as an HTTP response to the upgrade:
| Status | Code | Meaning |
|---|---|---|
| 400 | missing_language / invalid_language | No language, or not a short code. |
| 400 | invalid_vad_end_silence / invalid_auto_switch | A query parameter is out of range. |
| 401 | missing_api_key / invalid_api_key | No key, or not a valid one. |
| 402 | insufficient_credits / spend_limit_exceeded | Top up, or raise the project's spend limit. |
| 403 | insufficient_permissions / language_unavailable | The key lacks naad, or the language is not available. |
| 429 | too_many_sessions / rate_limit_exceeded | Concurrency or connect-rate limit. Honour Retry-After. |
After connecting, a language with no free capacity sends an error with language_at_capacity and closes the socket with code 4429 — back off before reconnecting.
Billing & limits
Billed at the speech-to-text rate (naad-stt-v2) for every second the socket is open, metered in one-minute segments plus a final one on close — not at the realtime agent rate. Sessions count against the same concurrency limits as realtime calls, per key, per account and per language, and close after 120 minutes.