Skip to main content

Documentation Index

Fetch the complete documentation index at: https://assemblyai.com/docs/llms.txt

Use this file to discover all available pages before exploring further.


Transcribe audio containing multiple languages with code switching detection. This feature enables accurate transcription of conversations where speakers naturally switch between languages during conversations.

Built-in code switching

Universal-3 Pro has code switching capabilities across English, Spanish, Portuguese, French, German, and Italian. To enable code switching, set the following parameter: "prompt": "The spoken language may change throughout the audio, transcribe in the original language mix (code-switching), preserving the words in the language they are spoken." Here is an example of how Universal-3 Pro handles spoken audio in English and French. Example output:
wordWrap
You literally lost your French? No, no, no. Mon français est là. L'italien, j'ai oublié, mais mon français est toujours là. Il partira jamais. Okay, but would you need a French coach? Could you consider having me? Oh yeah, yeah, absolutely, absolutely. Mais pour l'instant, le français est là, heureusement.
import requests
import time

base_url = "https://api.assemblyai.com"
headers = {"authorization": "<YOUR_API_KEY>"}

data = {
    "audio_url": "https://assemblyaiassets.com/audios/code_switching_multilingual.mp3",
    "language_detection": True,
    "speech_models": ["universal-3-pro", "universal-2"],
    "prompt": "The spoken language may change throughout the audio, transcribe in the original language mix (code-switching), preserving the words in the language they are spoken."
}

response = requests.post(base_url + "/v2/transcript", headers=headers, json=data)

if response.status_code != 200:
    print(f"Error: {response.status_code}, Response: {response.text}")
    response.raise_for_status()

transcript_response = response.json()
transcript_id = transcript_response["id"]
polling_endpoint = f"{base_url}/v2/transcript/{transcript_id}"

while True:
    transcript = requests.get(polling_endpoint, headers=headers).json()
    if transcript["status"] == "completed":
        print(transcript["text"])
        break
    elif transcript["status"] == "error":
        raise RuntimeError(f"Transcription failed: {transcript['error']}")
    else:
        time.sleep(3)
Language supportUniversal-3 Pro supports English, Spanish, Portuguese, French, German, and Italian. To access all 99 languages, use "speech_models": ["universal-3-pro", "universal-2"] as shown in the code example. Read more here.

Universal-2

Supports code switching between 99 languages, with best performing code switching languages being en, es, de. To enable code switching for Universal-2 only, set speech_models to universal-2, set language_detection to True, and set code_switching to True inside the language_detection_options parameter as shown in the example below.
If your use case involves code-switching between English, Spanish, Portuguese, French, German, or Italian, we recommend upgrading to Universal-3 Pro. It delivers significantly improved accuracy when speakers alternate between these languages mid-speech — outperforming Universal-2 across all supported language pairs.
import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
    speech_models=["universal-2"],
    language_detection=True,
    language_detection_options=aai.LanguageDetectionOptions(
      code_switching=True
    )
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

Example API Response

When enabling code switching with automatic language detection, the two detected language codes with the highest confidence and their confidence will be included in the transcript JSON.
"language_detection_results": {
      "code_switching_languages": [
           {"language": "en", "confidence": 0.8},
           {"language": "es", "confidence": 0.7}
     ]
}

99 languages coverage

To get the best performing transcription while extending code switching across all 99 supported languages, specify both “Universal-3 Pro” and “Universal-2” using the speech_models parameter, allowing our system to automatically route your audio based on language support. Model routing behavior: The system attempts to use the models in priority order falling back to the next model when needed. For example, with ["universal-3-pro", "universal-2"], the system will try to use universal-3-pro for languages it supports (English, Spanish, Portuguese, French, German, and Italian), and automatically fall back to Universal-2 for all other languages. This ensures you get the best performing transcription where available while maintaining the widest language coverage.

Other supported languages

While additional languages are supported for code switching, optimal results typically require the non-English language to be dominant in the audio. For English-dominant content with other languages, standard single-language transcription may be more appropriate. We highly recommend testing sample code switching files with your specific audio to assess performance and evaluate outputs. We also recommend using an LLM to correct and fine-tune our model’s outputs.

Manually Setting Language Codes

To manually set the language codes, you can use the language_codes parameter. A max of two language codes can be set and one code must be "en". For example, if your file contains both English and Spanish, it would be "language_codes": ["en", "es"].
import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
  speech_models=["universal-3-pro", "universal-2"],
  language_codes=["en", "es"]  # English-Spanish code switching
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

Code Switching Confidence Threshold

The code_switching_confidence_threshold parameter controls how the model routes transcription when multiple languages are detected. When code switching is enabled, the model detects up to two languages per audio file and assigns each a confidence score.
Code Switching Routing BehaviorThis parameter controls routing, not rejection. Audio is always transcribed, even if confidence scores do not meet this threshold. To return an error instead of a low-confidence transcription, you can use language_confidence_threshold alongside this parameter.
The threshold determines which language is used for transcription using the following logic:
  • If the non-English language’s confidence score meets or exceeds the threshold, the audio is routed to that non-English language model.
  • If the non-English language’s confidence score falls below the threshold, the audio is routed to whichever language has the highest overall confidence, which may be English or non-English.
  • If both detected languages are non-English, the audio is always routed to whichever has the higher confidence score, regardless of the threshold.
Code Switching DefaultBy default, the code_switching_confidence_threshold parameter is set to 0.3. If you would like to disable this, make sure to set this parameter to 0. Setting code_switching_confidence_threshold to 0 means the non-English language is always used for routing, even with very low confidence.
import assemblyai as aai

aai.settings.api_key = "<YOUR_API_KEY>"

audio_file = "./bilingual-audio.mp3"
# audio_file = "https://assembly.ai/wildfires.mp3"

config = aai.TranscriptionConfig(
    speech_models=["universal-3-pro", "universal-2"],
    language_detection=True,
    language_detection_options=aai.LanguageDetectionOptions(
      code_switching=True,
      code_switching_confidence_threshold=0.5 # Optional parameter - this is set to 0.3 by default
    )
)

transcript = aai.Transcriber(config=config).transcribe(audio_file)

if transcript.status == "error":
  raise RuntimeError(f"Transcription failed: {transcript.error}")

print(transcript.text)

Troubleshooting

If your audio contains primarily one language with only occasional words from another language, standard single-language transcription may be more appropriate than code-switching mode.