Gradium STT API
Speech-to-Text endpoints for converting audio to text
OpenAPI Specification
openapi: 3.1.0
info:
title: Gradium metering STT API
description: "This documentation covers the Gradium API.\n\nThis API exposes our Text-To-Speech and Speech-To-Text models, which offers low-latency, high-quality & natural sounding output and best in class accuracy. \n\nFor issues, questions, or feature requests, please contact us at support@gradium.ai"
version: 0.1.0
servers:
- url: https://api.gradium.ai/api
description: Gradium API
tags:
- name: STT
description: Speech-to-Text endpoints for converting audio to text
paths:
/speech/asr:
get:
tags:
- STT
summary: STT WebSocket Stream
description: "Connect to this endpoint via WebSocket for real-time speech-to-text conversion with streaming audio input.\n\n**Connection URL:**\n\n```\nwss://api.gradium.ai/api/speech/asr\n```\n\n**Authentication:**\nInclude your API key in the WebSocket connection header:\n- Header: `x-api-key: your_api_key`\n\n---\n\n## Quick Reference\n\n| Direction | Message Type | Example |\n|-----------|-------------|---------|\n| \U0001F535⬆️ Client→Server | Setup (first) | `{\"type\": \"setup\", \"model_name\": \"default\", \"input_format\": \"pcm\", \"json_config\": {\"language\": \"en\", \"delay_in_frames\": 16}}` |\n| \U0001F7E2⬇️ Server→Client | Ready | `{\"type\": \"ready\", \"request_id\": \"uuid\", \"model_name\": \"default\", \"sample_rate\": 24000}` |\n| \U0001F535⬆️ Client→Server | Audio | `{\"type\": \"audio\", \"audio\": \"base64...\"}` |\n| \U0001F7E2⬇️ Server→Client | Text (result) | `{\"type\": \"text\", \"text\": \"Hello world\", \"start_s\": 0.5}` |\n| \U0001F7E2⬇️ Server→Client | VAD (activity) | `{\"type\": \"step\", \"vad\": [...], \"step_idx\": 5, \"step_duration_s\": 0.08}` |\n| \U0001F7E2⬇️ Server→Client | End Text | `{\"type\": \"end_text\", \"stop_s\": 2.5}` |\n| \U0001F535⬆️ Client→Server | Flush | `{\"type\": \"flush\", \"flush_id\": 1}` |\n| \U0001F7E2⬇️ Server→Client | Flushed | `{\"type\": \"flushed\", \"flush_id\": 1}` |\n| \U0001F535⬆️ Client→Server | EndOfStream | `{\"type\": \"end_of_stream\"}` |\n| \U0001F7E2⬇️ Server→Client | EndOfStream | `{\"type\": \"end_of_stream\"}` |\n| \U0001F534⬇️ Server→Client | Error | `{\"type\": \"error\", \"message\": \"Error description\", \"code\": 1008}` |\n\n---\n\n## Message Types\n\n### 1. Setup Message (First Message)\n\n**Direction:** Client → Server\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"setup\",\n \"model_name\": \"default\",\n \"input_format\": \"pcm\",\n \"json_config\": {\n \"language\": \"en\",\n \"delay_in_frames\": 16\n }\n}\n```\n\n**Fields:**\n- `type` (string, required): Must be \"setup\"\n- `model_name` (string, optional): The Speech-To-Text model to use (default: \"default\")\n- `input_format` (string, optional): Audio format (default: \"wav\"). One of \"pcm\", \"pcm_8000\", \"pcm_16000\", \"pcm_22050\", \"pcm_24000\", \"pcm_44100\", \"pcm_48000\", \"wav\", \"opus\", \"ulaw_8000\", \"mulaw_8000\", \"alaw_8000\".\n- `json_config` (object or string, optional): Advanced STT settings, for example `{\"language\":\"en\",\"delay_in_frames\":16}`.\n\n**Important:** This must be the very first message sent after connection. The server will close the connection if any other message is sent first.\n\n---\n\n### 2. Ready Message\n\n**Direction:** Server → Client\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"ready\",\n \"request_id\": \"550e8400-e29b-41d4-a716-446655440000\",\n \"model_name\": \"default\",\n \"sample_rate\": 24000,\n \"frame_size\": 1920,\n \"delay_in_frames\": 0,\n \"text_stream_names\": []\n}\n```\n\n**Fields:**\n- `type` (string): Will be \"ready\"\n- `request_id` (string): Unique identifier for the session\n- `model_name` (string): The Speech To Text model being used\n- `sample_rate` (integer): Expected sample rate in Hz (typically 24000)\n- `frame_size` (int): Number of samples by which the model processes data (typically 1920 which is equivalent to 80ms at 24kHz)\n- `delay_in_frames` (integer): Delay in audio frames for the model\n- `text_stream_names` (array): List of text stream names\n\nThis message is sent by the server after receiving the setup message, indicating that the connection is ready to receive audio.\n\n---\n\n### 3. Audio Message\n\n**Direction:** Client → Server\n**Format:** JSON Object (with binary audio data)\n\n```json\n{\n \"type\": \"audio\",\n \"audio\": \"base64_encoded_audio_data...\"\n}\n```\n\n**Fields:**\n- `type` (string, required): Must be \"audio\"\n- `audio` (string, required): Base64-encoded audio data\n\n**Audio Format Requirements (for PCM input):**\n- **Sample Rate**: 24000 Hz (24kHz)\n- **Format**: PCM (Pulse Code Modulation)\n- **Bit Depth**: 16-bit signed integer (little-endian)\n- **Channels**: Single channel (mono)\n- **Chunk Size**: Recommended 1920 samples per frame (80ms at 24kHz)\n\nWhen using `\"wav\"` input format, the audio must be a valid WAV file using\nPCM data (so `AudioFormat` = 1 in the WAV header). Supported bits per sample\nare 16, 24 and 32 bits.\n\nWhen using `\"opus\"` input format, the audio must be some ogg wrapped opus data\nstream.\n\nSend audio messages to be transcribed. You can send multiple audio messages in sequence. The server will stream text and VAD responses as it processes the audio.\n\n---\n\n### 4. Text Response\n\n**Direction:** Server → Client\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"text\",\n \"text\": \"Hello world\",\n \"start_s\": 0.5,\n \"stream_id\": 0\n}\n```\n\n**Fields:**\n- `type` (string): Will be \"text\"\n- `text` (string): The transcribed text\n- `start_s` (float): Start time of the transcription in seconds\n- `stream_id` (integer or null): Stream identifier for tracking multiple concurrent streams\n\nText messages contain the transcribed speech. Multiple text messages will be streamed as the audio is processed.\n\n---\n\n### 5. VAD Response (Voice Activity Detection)\n\n**Direction:** Server → Client\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"step\",\n \"vad\": [\n {\n \"horizon_s\": 0.5,\n \"inactivity_prob\": 0.05\n },\n {\n \"horizon_s\": 1.0,\n \"inactivity_prob\": 0.08\n },\n {\n \"horizon_s\": 2.0,\n \"inactivity_prob\": 0.12\n }\n ],\n \"step_idx\": 5,\n \"step_duration_s\": 0.08,\n \"total_duration_s\": 0.4\n}\n```\n\n**Fields:**\n- `type` (string): Will be \"step\"\n- `vad` (array): List of VAD predictions with future horizons\n - `horizon_s` (float): Lookahead duration in seconds\n - `inactivity_prob` (float): Probability that voice activity has ended by this horizon in seconds.\n- `step_idx` (integer): The step index (increments every 80ms)\n- `step_duration_s` (float): Duration of this step in seconds (typically 0.08)\n- `total_duration_s` (float): Total duration of audio processed so far\n\n**VAD Interpretation:**\n- VAD messages are emitted every 80ms (one per audio frame)\n- Use the `inactivity_prob` value from the longest horizon to determine if the speaker has likely finished\n- Higher `inactivity_prob` values indicate higher confidence that speaking has ended\n- Recommended threshold: Use `vad[2][\"inactivity_prob\"]` (third prediction) as the turn-taking indicator\n\n---\n\n### 6. End Text Response\n\n**Direction:** Server → Client\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"end_text\",\n \"stop_s\": 2.5,\n \"stream_id\": 0\n}\n```\n\n**Fields:**\n- `type` (string): Will be \"end_text\"\n- `stop_s` (float): Stop time of last `text` message in seconds\n- `stream_id` (integer or null): Stream identifier\n\nSent when the previous text segment has a finished and its end timestamp is\navailable.\n\n---\n\n### 7. Flush Message\n\n**Direction:** Client → Server\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"flush\",\n \"flush_id\": 1\n}\n```\n\n**Fields:**\n- `type` (string, required): Must be \"flush\"\n- `flush_id` (integer, required): Identifier for this flush request, echoed back in the `flushed` reply.\n\nThis message can be sent by the client to request the server to flush any\nbuffered audio and return all outstanding text results immediately. The server\nwill respond with a `flushed` message containing the same `flush_id` once the\nflush is complete.\n\n### 8. End Of Stream\n\n**Direction:** Client → Server and Server → Client\n**Format:** JSON Object\n\n```json\n{\n \"type\": \"end_of_stream\"\n}\n```\n\nThis message is sent by the client when it has finished sending audio. The server will then process any remaining audio and send back all outstanding text results, VAD information, and then an `end_of_stream` message before closing the connection.\n\n---\n\n## Error Handling\n\nWhen errors occur, the server sends an error message as JSON before closing the connection:\n\n**Error Message Format:**\n```json\n{\n \"type\": \"error\",\n \"message\": \"Error description explaining what went wrong\",\n \"code\": 1008\n}\n```\n\n**Common Error Codes:**\n- `1008`: Policy Violation (e.g., invalid API key, missing setup message, invalid audio format)\n- `1011`: Internal Server Error (unexpected server-side error)\n\n---\n\n## Best Practices for STT\n\n1. **Always send setup first**: The server expects a setup message immediately after connection\n2. **Use correct audio format**: When using PCM, ensure audio is 24kHz PCM 16-bit mono\n3. **Send appropriately sized chunks**: 1920 samples (80ms) per message is recommended\n4. **Graceful shutdown**: Send `end_of_stream` when done to properly close the session\n5. **VAD Threshold**: Our VAD provides estimated probabilities that the speaker would be silent for a fixed number of seconds in the future. The thresholds to trigger the end-of-the-turn decisions might be application-dependent; as a starting point we recommend looking at the horizon of 2s and trigger when the inactivity_prob is above 0.5: `turn_ended = msg[\"vad\"][2][\"inactivity_prob\"] > 0.5`.\n5. **Acting on VAD**: Whenever you decide that the VAD probabilities warrant a decision to consider the turn ended, there is still up to `delay_in_frames` audio frames processed by the model. Instead of feeding silence from the speaker, the system can be made more reactive by flushing the remainder of the turn's transcript. For that, you can feed in `delay_in_frames` chunks of silence (vectors of zeros). If those are fed in faster than realtime, the API also has a possibility to process them faster, allowing a considerably more reactive turn-around.\n"
parameters:
- name: x-api-key
in: header
required: true
schema:
type: string
description: Your Gradium API key
responses:
'101':
description: WebSocket connection established
x-codeSamples:
- lang: cURL
source: "wscat -c \"wss://api.gradium.ai/api/speech/asr\" \\\n -H \"x-api-key: your_api_key\"\n# After connection, paste:\n# {\"type\":\"setup\",\"model_name\":\"default\",\"input_format\":\"pcm\",\"json_config\":{\"language\":\"en\",\"delay_in_frames\":16}}\n"
- lang: Python
source: "import asyncio\nimport base64\nimport json\n\nimport websockets\n\nCHUNK_BYTES = 1920 * 2 # 80 ms at 24 kHz, 16-bit mono.\n\n\nasync def transcribe(api_key: str, pcm_audio: bytes):\n setup = {\n \"type\": \"setup\",\n \"model_name\": \"default\",\n \"input_format\": \"pcm\",\n \"json_config\": {\n \"language\": \"en\",\n \"delay_in_frames\": 16,\n },\n }\n\n async with websockets.connect(\n \"wss://api.gradium.ai/api/speech/asr\",\n additional_headers={\"x-api-key\": api_key},\n ) as ws:\n await ws.send(json.dumps(setup))\n ready = json.loads(await ws.recv())\n assert ready[\"type\"] == \"ready\"\n\n async def producer():\n for off in range(0, len(pcm_audio), CHUNK_BYTES):\n chunk = pcm_audio[off : off + CHUNK_BYTES]\n await ws.send(json.dumps({\n \"type\": \"audio\",\n \"audio\": base64.b64encode(chunk).decode(),\n }))\n await ws.send(json.dumps({\"type\": \"end_of_stream\"}))\n\n async def consumer():\n while True:\n msg = json.loads(await ws.recv())\n if msg[\"type\"] == \"text\":\n print(msg[\"text\"])\n elif msg[\"type\"] == \"end_of_stream\":\n return\n elif msg[\"type\"] == \"error\":\n raise RuntimeError(msg[\"message\"])\n\n await asyncio.gather(producer(), consumer())\n\n\nasyncio.run(transcribe(\"your_api_key\", open(\"input.pcm\", \"rb\").read()))\n"
/post/speech/asr:
post:
tags:
- STT
summary: STT POST Endpoint
description: "Use this HTTP POST endpoint for simple, one-shot speech-to-text\ntranscription. Send the entire audio payload in the request body and receive\na stream of newline-delimited JSON (NDJSON) messages with the transcription\nresults.\n\n**Endpoint URL:**\n\n```\nhttps://api.gradium.ai/api/post/speech/asr\n```\n\n**Authentication:**\nInclude your API key in the request header:\n- Header: `x-api-key: your_api_key`\n\n---\n\n## Quick Example\n\n```bash\ncurl -L -X POST https://api.gradium.ai/api/post/speech/asr \\\n -H \"x-api-key: your_api_key\" \\\n -H \"Content-Type: audio/wav\" \\\n --data-binary @input.wav\n```\n\nWith a language hint:\n\n```bash\ncurl -L -X POST \"https://api.gradium.ai/api/post/speech/asr?json_config=%7B%22language%22%3A%22en%22%7D\" \\\n -H \"x-api-key: your_api_key\" \\\n -H \"Content-Type: audio/wav\" \\\n --data-binary @input.wav\n```\n\n---\n\n## Request Format\n\n**Method:** POST\n**Body:** Raw audio bytes (the full file).\n\nThe input audio format is selected from the `Content-Type` header:\n\n| Content-Type | Audio Format |\n|--------------|--------------|\n| `audio/wav` (default if header is missing) | WAV (PCM data, 16/24/32-bit) |\n| `audio/pcm` | Raw PCM, 24 kHz, 16-bit signed little-endian, mono |\n| `audio/ogg` or `audio/opus` | Ogg-wrapped Opus |\n\n**Query Parameters:**\n- `model` (string, optional): The Speech-to-Text model to use (default: `default`).\n- `input_format` (string, optional): Override the input format detected from\n `Content-Type`. One of `wav`, `pcm`, `opus`.\n- `json_config` (string, optional): JSON-encoded model configuration. Common\n use case: pass a language hint, e.g. `{\"language\": \"en\"}`. The value should\n be URL-encoded when used as a query parameter.\n\n---\n\n## Response Format\n\n**Content-Type:** `application/x-ndjson`\n\nThe response body is a stream of newline-delimited JSON messages. Each line\nis a separate JSON object. Possible message types:\n\n### `text` — transcribed text segment\n\n```json\n{\"type\": \"text\", \"text\": \"Hello world\", \"start_s\": 0.5, \"stream_id\": 0}\n```\n\n- `text` (string): Transcribed text.\n- `start_s` (float): Start time of the segment in seconds.\n- `stream_id` (integer): Stream identifier when multiple text streams are in\n use (0 in single-stream transcription).\n\n### `end_text` — segment boundary\n\n```json\n{\"type\": \"end_text\", \"stop_s\": 2.5, \"stream_id\": 0}\n```\n\n- `stop_s` (float): End time of the previous `text` segment in seconds.\n- `stream_id` (integer): Stream identifier.\n\n### `error` — server-side error\n\n```json\n{\"type\": \"error\", \"message\": \"Error description\"}\n```\n\nIf the transcription pipeline fails, the server emits an `error` message and\nstops the stream.\n\n---\n\n## Reading the Stream\n\nThe response is streamed: read the body line-by-line and parse each line as\nJSON. The body closes when transcription is complete.\n\n```python\nimport json\nimport requests\n\nwith open(\"input.wav\", \"rb\") as f:\n audio = f.read()\n\nwith requests.post(\n \"https://api.gradium.ai/api/post/speech/asr\",\n data=audio,\n headers={\n \"x-api-key\": \"your_api_key\",\n \"Content-Type\": \"audio/wav\",\n },\n stream=True,\n) as resp:\n resp.raise_for_status()\n transcript = []\n for line in resp.iter_lines(decode_unicode=True):\n if not line:\n continue\n msg = json.loads(line)\n if msg[\"type\"] == \"text\":\n transcript.append(msg[\"text\"])\n elif msg[\"type\"] == \"error\":\n raise RuntimeError(msg[\"message\"])\nprint(\" \".join(transcript))\n```\n\n---\n\n## Error Handling\n\nIf the request fails before the response stream has started, the server\nresponds with `HTTP 500` and a plain-text body. Two body shapes occur:\n\n- **Upstream errors** (with a numeric code) such as authentication\n failures or worker-level rejections:\n\n ```\n error from server <code>: <reason>\n ```\n\n For example, a revoked or expired API key returns\n `error from server 1008: API key is revoked or expired`.\n\n- **Proxy-level rejections** (e.g. unsupported `Content-Type`, malformed\n request body) come back as raw error strings without the `error from\n server` prefix:\n\n ```\n unsupported content type for SST 'audio/mpeg'\n ```\n\nIn both cases the body is plain text (not JSON). Errors that occur\nafter the NDJSON stream has started are surfaced as\n`{\"type\": \"error\", \"message\": \"...\"}` lines within the stream rather\nthan as a different HTTP status.\n\n---\n\n## When to Use POST vs WebSocket\n\nThe POST endpoint is ideal for one-shot transcription of complete audio\nfiles already on disk or in memory. The audio is uploaded in a single\nrequest, transcription runs, and the results are streamed back as NDJSON.\n\nUse the [WebSocket endpoint](/api-reference/endpoint/stt-websocket) instead\nwhen you need to:\n- Stream audio as it is being captured (microphone, telephony).\n- Receive partial transcripts and Voice Activity Detection (VAD) events in\n real time for turn-taking.\n- Send a `flush` message to force the model to emit buffered text on demand.\n"
parameters:
- name: x-api-key
in: header
required: true
schema:
type: string
description: Your Gradium API key
- name: Content-Type
in: header
required: false
schema:
type: string
enum:
- audio/wav
- audio/pcm
- audio/ogg
- audio/opus
description: Format of the audio in the request body. Defaults to audio/wav when omitted.
- name: model
in: query
required: false
schema:
type: string
default: default
description: Speech-to-Text model name.
- name: input_format
in: query
required: false
schema:
type: string
enum:
- wav
- pcm
- opus
description: Overrides the audio format detected from Content-Type.
- name: json_config
in: query
required: false
schema:
type: string
description: 'JSON-encoded model configuration. Example: {"language": "en"}'
requestBody:
required: true
content:
audio/wav:
schema:
type: string
format: binary
description: WAV audio file.
audio/pcm:
schema:
type: string
format: binary
description: 'Raw PCM audio: 24 kHz, 16-bit signed little-endian, mono.'
audio/ogg:
schema:
type: string
format: binary
description: Ogg-wrapped Opus audio.
responses:
'200':
description: NDJSON stream of transcription messages.
content:
application/x-ndjson:
schema:
type: string
description: 'Newline-delimited JSON messages: text, end_text, or error. The body closes when transcription is complete.'
'500':
description: 'Pre-stream error. Body is plain text. Upstream errors (authentication, worker rejections) are formatted as `error from server <code>: <reason>`; proxy-level rejections (e.g. unsupported Content-Type) come back as raw error strings.'
x-codeSamples:
- lang: cURL
source: "curl -L -X POST https://api.gradium.ai/api/post/speech/asr \\\n -H \"x-api-key: your_api_key\" \\\n -H \"Content-Type: audio/wav\" \\\n --data-binary @input.wav\n"
- lang: Python
source: "import json\n\nimport requests\n\nwith open(\"input.wav\", \"rb\") as f:\n audio = f.read()\n\nwith requests.post(\n \"https://api.gradium.ai/api/post/speech/asr\",\n data=audio,\n headers={\n \"x-api-key\": \"your_api_key\",\n \"Content-Type\": \"audio/wav\",\n },\n stream=True,\n) as resp:\n resp.raise_for_status()\n for line in resp.iter_lines(decode_unicode=True):\n if not line:\n continue\n msg = json.loads(line)\n if msg[\"type\"] == \"text\":\n print(msg[\"text\"])\n"
x-tagGroups:
- name: Documentation
tags:
- Documentation
- FAQ
- Release notes
- name: API Reference
tags:
- TTS
- STT
- Voices
- Pronunciations
- Credits