{
"type": "<string>",
"id": "<string>",
"expires_at": 123
}{
"type": "Turn",
"turn_order": 0,
"turn_is_formatted": true,
"end_of_turn": true,
"transcript": "Hello world.",
"end_of_turn_confidence": 0.98,
"words": [
{
"text": "Hello",
"start": 0,
"end": 500,
"confidence": 0.99
},
{
"text": "world.",
"start": 500,
"end": 1000,
"confidence": 0.98
}
]
}{
"type": "<string>",
"audio_duration_seconds": 123,
"session_duration_seconds": 123
}{
"type": "LLMGatewayResponse",
"turn_order": 0,
"transcript": "Hello world.",
"data": {
"request_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I help?"
},
"finish_reason": "stop"
}
],
"usage": {
"input_tokens": 12,
"output_tokens": 8,
"total_tokens": 20,
"prompt_tokens_details": {},
"completion_tokens_details": {}
},
"request": {},
"response_time": 123456789
}
}"\\x10\\x00\\x20\\x00\\x30\\x00\\x40\\x00\\x30\\x00\\x20\\x00\\x10\\x00\\x00\\x00\\xf0\\xff\\xe0\\xff\\xd0\\xff\\xc0\\xff"{
"type": "UpdateConfiguration",
"sample_rate": 16000,
"encoding": "pcm_s16le",
"format_turns": true,
"keyterms_prompt": [
"AssemblyAI",
"Universal Streaming"
]
}{
"type": "ForceEndpoint"
}{
"type": "<string>"
}{
"type": "KeepAlive"
}Universal Streaming
Stream audio and receive real-time transcription results. Fast, cost-effective streaming transcription available in three variants:
- Universal-Streaming English — the fastest real-time English transcription
- Universal-Streaming Multilingual — multilingual support (English, Spanish, German, French, Portuguese, and Italian) at the same speed and price
- Whisper-Streaming — open-source Whisper powered by AssemblyAI’s infrastructure with 99+ languages
To use the EU server for Streaming STT, replace streaming.assemblyai.com with
streaming.eu.assemblyai.com.
Documentation Index
Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt
Use this file to discover all available pages before exploring further.
{
"type": "<string>",
"id": "<string>",
"expires_at": 123
}{
"type": "Turn",
"turn_order": 0,
"turn_is_formatted": true,
"end_of_turn": true,
"transcript": "Hello world.",
"end_of_turn_confidence": 0.98,
"words": [
{
"text": "Hello",
"start": 0,
"end": 500,
"confidence": 0.99
},
{
"text": "world.",
"start": 500,
"end": 1000,
"confidence": 0.98
}
]
}{
"type": "<string>",
"audio_duration_seconds": 123,
"session_duration_seconds": 123
}{
"type": "LLMGatewayResponse",
"turn_order": 0,
"transcript": "Hello world.",
"data": {
"request_id": "3c90c3cc-0d44-4b50-8888-8dd25736052a",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "Hello! How can I help?"
},
"finish_reason": "stop"
}
],
"usage": {
"input_tokens": 12,
"output_tokens": 8,
"total_tokens": 20,
"prompt_tokens_details": {},
"completion_tokens_details": {}
},
"request": {},
"response_time": 123456789
}
}"\\x10\\x00\\x20\\x00\\x30\\x00\\x40\\x00\\x30\\x00\\x20\\x00\\x10\\x00\\x00\\x00\\xf0\\xff\\xe0\\xff\\xd0\\xff\\xc0\\xff"{
"type": "UpdateConfiguration",
"sample_rate": 16000,
"encoding": "pcm_s16le",
"format_turns": true,
"keyterms_prompt": [
"AssemblyAI",
"Universal Streaming"
]
}{
"type": "ForceEndpoint"
}{
"type": "<string>"
}{
"type": "KeepAlive"
}The speech model used for your Streaming session.
universal-streaming-english, universal-streaming-multilingual, whisper-rtUse your API key for authentication, or alternatively generate a temporary token and pass it via the token query parameter.
Encoding of the audio stream.
pcm_s16le, pcm_mulawWhether to return formatted final transcripts.
true, falseOptional time in seconds of inactivity before session is terminated (integer, minimum 5, maximum 3600). If not set, no inactivity timeout is applied.
A list of words and phrases to improve recognition accuracy for. See Keyterms Prompting for more details.
Whether to detect the language and return language metadata on utterances and final turns. Only available for the multilingual model.
true, falseThe maximum amount of silence in milliseconds allowed in a turn before end of turn is triggered. See Turn Detection for configuration details.
The minimum amount of silence in milliseconds required to detect end of turn when confident. See Turn Detection for configuration details.
Sample rate of the audio stream.
Whether to enable Streaming Speaker Diarization. When enabled, each Turn event will include a speaker_label field and each final word in the words array will include a speaker field for word-level speaker attribution.
true, falseThe maximum number of speakers expected in the audio stream (integer, 1-10). Setting this can improve speaker label accuracy when you know the number of speakers in advance. Only used when speaker_labels is enabled. See Streaming Diarization for more details.
API token for authentication (if using a temporary token).
The confidence threshold (0.0 to 1.0) for classifying audio frames as silence. Frames with VAD confidence below this value are considered silent. Increase for noisy environments to reduce false speech detection.
The confidence threshold (0.0 to 1.0) to use when determining if the end of a turn has been reached. See Turn Detection for configuration details.
Note: This parameter is only supported for the Universal-streaming model.
Enable domain-specific transcription models to improve accuracy for specialized terminology. Set to "medical-v1" to enable Medical Mode for improved accuracy of medical terms such as medications, procedures, conditions, and dosages. Supported languages: English (en), Spanish (es), German (de), French (fr). If used with an unsupported language, the parameter is ignored and a warning is returned.
medical-v1The language of your audio stream. Deprecated.
en, multiJSON-stringified LLM Gateway configuration that processes each finalized turn. Follows the same interface as the Chat Completions endpoint and accepts model, messages, tools, tool_choice, post_processing_steps, and max_tokens. See Apply LLM Gateway to Streaming for the full schema and examples.
Server message indicating the streaming session has successfully started.
Server message containing a formatted turn-based transcription result.
Server message confirming session termination with session statistics.
Server message containing an LLM Gateway response for a finalized turn.
Client sends audio data as raw binary.
Client message to update streaming configuration parameters during an active session.
Client message to manually force an endpoint in the transcription.
Client message to gracefully terminate the streaming session.
Client message to reset the inactivity timeout timer. This is not necessary by default — sessions remain open until explicitly terminated or until the 3-hour maximum session duration is reached. This message is only needed if you have set inactivity_timeout and want to keep the session open during periods where no audio is being sent.
Was this page helpful?