LLM Gateway Overview

Documentation Index

Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt

Use this file to discover all available pages before exploring further.

​

Overview

• Basic Chat Completions - Simple request/response interactions
• Streamed Responses - Stream output as it’s generated (OpenAI models)
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain responses to a specific JSON schema
• Tool/Function Calling - Enable models to execute custom functions
• Agentic Workflows - Multi-step reasoning with automatic tool chaining
• Unified Interface - One API for Claude, GPT, Gemini, Qwen, Kimi, and more
• Post-processing - Automatically repair malformed JSON responses with built-in JSON repair

​

Available models

​

By quality (LMArena Score)

​

By latency (per 10,000 tokens)

​

By provider

​

Anthropic Claude

​

OpenAI GPT

​

Google Gemini

​

Alibaba Cloud Qwen

​

Moonshot AI Kimi

​

Select a model

import requests

headers = {
  "authorization": "<YOUR_API_KEY>"
}

response = requests.post(
    "https://llm-gateway.assemblyai.com/v1/chat/completions",
    headers = headers,
    json = {
        "model": "claude-sonnet-4-6",
        "messages": [
            {"role": "user", "content": "What is the capital of France?"}
        ],
        "max_tokens": 1000
    }
)

result = response.json()
print(result["choices"][0]["message"]["content"])

const response = await fetch(
  "https://llm-gateway.assemblyai.com/v1/chat/completions",
  {
    method: "POST",
    headers: {
      authorization: "<YOUR_API_KEY>",
      "content-type": "application/json",
    },
    body: JSON.stringify({
      model: "claude-sonnet-4-6",
      messages: [{ role: "user", content: "What is the capital of France?" }],
      max_tokens: 1000,
    }),
  }
);

const result = await response.json();
console.log(result.choices[0].message.content);

​

Logging and troubleshooting

• request_id from the response body
• The model parameter used
• The API region (US: llm-gateway.assemblyai.com, EU: llm-gateway.eu.assemblyai.com)
• A timestamp for when the request was sent
• The full error response body when a non-2xx status code is returned

​

Next steps

• Basic Chat Completions - Learn how to send simple messages and receive responses
• Multi-turn Conversations - Maintain context across multiple exchanges
• Structured Outputs - Constrain model responses to follow a specific JSON schema
• Tool Calling - Enable models to execute custom functions
• Agentic Workflows - Build multi-step reasoning applications
• Post-processing - Automatically repair malformed JSON in model responses