Creating summarized chapters from podcasts

The Auto Chapters approach uses LLM Gateway to summarize audio data over time into chapters. Chapters make it easy for users to navigate and find specific information. Each chapter contains the following:

Summary
One-line gist
Headline
Start and end timestamps

The auto_chapters parameter on the transcription API is deprecated. Use LLM Gateway as shown below for more flexible and powerful chapter summaries.

In this step-by-step guide, you’ll learn how to generate chapter summaries using LLM Gateway. You’ll transcribe your audio, retrieve the paragraphs, and then use LLM Gateway to generate chapter summaries.

Get started

Before we begin, make sure you have an AssemblyAI account and an API key. You can sign up for a free account and get your API key from your dashboard. Here’s an audio example for this guide:

https://assembly.ai/wildfires.mp3

Step-by-step instructions

Python
JavaScript

Create a new file and import the necessary libraries.

import requests
import time

Create a new file and import the necessary libraries.

import fs from "fs-extra";

Python
JavaScript

Set up the API endpoint and headers.

base_url = "https://api.assemblyai.com"

headers = {
    "authorization": "<YOUR_API_KEY>"
}

Set up the API endpoint and headers.

const baseUrl = "https://api.assemblyai.com";

const headers = {
  authorization: "<YOUR_API_KEY>",
};

Python
JavaScript

Upload your local file and submit a transcription request.

with open("./my-audio.mp3", "rb") as f:
  response = requests.post(base_url + "/v2/upload",
                          headers=headers,
                          data=f)

upload_url = response.json()["upload_url"]

data = {
    "audio_url": upload_url,
    "speech_models": ["universal-3-pro", "universal-2"],
    "language_detection": True
}

response = requests.post(base_url + "/v2/transcript", json=data, headers=headers)

transcript_id = response.json()['id']
polling_endpoint = base_url + "/v2/transcript/" + transcript_id

while True:
    transcription_result = requests.get(polling_endpoint, headers=headers).json()
    if transcription_result['status'] == 'completed':
        break
    elif transcription_result['status'] == 'error':
        raise RuntimeError(f"Transcription failed: {transcription_result['error']}")
    else:
        time.sleep(3)

Upload your local file and submit a transcription request.

const path = "./my-audio.mp3";
const audioData = await fs.readFile(path);
let res = await fetch(`${baseUrl}/v2/upload`, {
  method: "POST",
  headers,
  body: audioData,
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const uploadResponse = await res.json();
const uploadUrl = uploadResponse.upload_url;

const data = {
  audio_url: uploadUrl,
  speech_models: ["universal-3-pro", "universal-2"],
  language_detection: true,
};

res = await fetch(`${baseUrl}/v2/transcript`, {
  method: "POST",
  headers: { ...headers, "Content-Type": "application/json" },
  body: JSON.stringify(data),
});
if (!res.ok) throw new Error(`Error: ${res.status}`);
const response = await res.json();

const transcriptId = response.id;
const pollingEndpoint = `${baseUrl}/v2/transcript/${transcriptId}`;

let transcriptionResult;
while (true) {
  res = await fetch(pollingEndpoint, { headers });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  transcriptionResult = await res.json();

  if (transcriptionResult.status === "completed") {
    break;
  } else if (transcriptionResult.status === "error") {
    throw new Error(`Transcription failed: ${transcriptionResult.error}`);
  } else {
    await new Promise((resolve) => setTimeout(resolve, 3000));
  }
}

Python
JavaScript

Get paragraphs from the transcript and group them into chapters.

paragraphs = requests.get(polling_endpoint + '/paragraphs', headers=headers).json()['paragraphs']

# Group paragraphs into chapters
step = 2  # Adjust to control chapter length
combined_paragraphs = []

for i in range(0, len(paragraphs), step):
    paragraph_group = paragraphs[i : i + step]
    start = paragraph_group[0]['start']
    end = paragraph_group[-1]['end']
    text = " ".join(p['text'] for p in paragraph_group)
    combined_paragraphs.append({"text": text, "start": start, "end": end})

Get paragraphs from the transcript and group them into chapters.

let res = await fetch(`${pollingEndpoint}/paragraphs`, { headers });
if (!res.ok) throw new Error(`Error: ${res.status}`);
const paragraphsResponse = await res.json();
const { paragraphs } = paragraphsResponse;

// Group paragraphs into chapters
const step = 2; // Adjust to control chapter length
const combinedParagraphs = [];

for (let i = 0; i < paragraphs.length; i += step) {
  const group = paragraphs.slice(i, i + step);
  combinedParagraphs.push({
    text: group.map((p) => p.text).join(" "),
    start: group[0].start,
    end: group[group.length - 1].end,
  });
}

Python
JavaScript

Generate chapter summaries using LLM Gateway.

for chapter in combined_paragraphs:
    llm_gateway_data = {
        "model": "claude-sonnet-4-6",
        "messages": [
            {"role": "user", "content": f"Provide a brief one-paragraph summary, a one-line gist, and a headline for this section of a transcript.\n\nText: {chapter['text']}"}
        ],
        "max_tokens": 500
    }

    response = requests.post(
        "https://llm-gateway.assemblyai.com/v1/chat/completions",
        headers=headers,
        json=llm_gateway_data
    )

    result = response.json()["choices"][0]["message"]["content"]
    print(f"Chapter Start Time: {chapter['start']}")
    print(f"Chapter End Time: {chapter['end']}")
    print(f"Chapter Summary: {result}\n")

Generate chapter summaries using LLM Gateway.

for (const chapter of combinedParagraphs) {
  const llmGatewayData = {
    model: "claude-sonnet-4-6",
    messages: [
      {
        role: "user",
        content: `Provide a brief one-paragraph summary, a one-line gist, and a headline for this section of a transcript.\n\nText: ${chapter.text}`,
      },
    ],
    max_tokens: 500,
  };

  let res = await fetch("https://llm-gateway.assemblyai.com/v1/chat/completions", {
    method: "POST",
    headers: { ...headers, "Content-Type": "application/json" },
    body: JSON.stringify(llmGatewayData),
  });
  if (!res.ok) throw new Error(`Error: ${res.status}`);
  const result = await res.json();

  console.log("Chapter Start Time:", chapter.start);
  console.log("Chapter End Time:", chapter.end);
  console.log(
    "Chapter Summary:",
    result.choices[0].message.content,
    "\n"
  );
}

Understanding the response

The LLM Gateway returns a summary for each chapter group. You can customize the output format by adjusting the prompt. For example, you can request a JSON response:

prompt = """For this section of a transcript, provide the following in JSON format:
{
  "headline": "A single sentence headline",
  "gist": "A few words summarizing the section",
  "summary": "A one paragraph summary"
}

Text: """ + chapter['text']

Conclusion

Creating chapter summaries using LLM Gateway gives you full control over the format and content of each chapter. You can customize the prompt to match your needs, use different models, and even use Structured Outputs for consistent JSON formatting. This approach works on all kinds of input sources, not just podcasts. For example, you can use it to summarize lecture videos or other long-form content.

Next steps

Auto Chapters - Learn more about generating chapter summaries
LLM Gateway Overview - Explore all available models and features
Structured Outputs - Constrain responses to a specific JSON schema

Documentation Index

​Get started

​Step-by-step instructions

​Understanding the response

​Conclusion

​Next steps

Get started

Step-by-step instructions

Understanding the response

Conclusion

Next steps