How do I transcribe audio in languages like Spanish, French, or German?

‍Global teams regularly work with audio and video content containing multiple languages, creating challenges when you need accurate written records. Whether you're documenting international business meetings where participants speak Spanish, French, and German, or creating accessible content from multilingual media, converting this audio into searchable text requires specialized technology that can handle language switching and maintain speaker identification across different languages.

This guide explains how multilingual transcription works, from automatic language detection to choosing between AI and human services. You'll learn about the key technologies that make it possible, practical implementation strategies, and best practices for getting accurate results when working with Spanish, French, German, and other languages in your audio content.

What is multilingual transcription?

Multilingual transcription is converting spoken audio that contains multiple languages into written text. This means when you have a recording where people speak Spanish, French, German, or any combination of languages, the system writes down exactly what was said in each original language. AssemblyAI's approach uses two complementary models to handle this: Universal-3 Pro delivers the highest accuracy for English, Spanish, French, German, Portuguese, and Italian, while Universal-2 extends coverage to 99+ languages including Mandarin, Hindi, Arabic, Japanese, and dozens more.

Unlike translation, multilingual transcription doesn't change languages—it preserves them. If someone speaks French in your recording, you'll get French text back, not English.

If you do need translation after transcription, you can pipe your transcript through AssemblyAI's LLM Gateway, which provides access to 25+ leading LLMs from providers like Anthropic, OpenAI, and Google. A single API call can translate your multilingual transcript into any target language while preserving speaker labels and timestamps.

Automatic language detection

Your transcription system automatically figures out which language is being spoken without you having to tell it. This means you can upload a file where someone starts speaking Spanish, switches to English mid-sentence, then continues in German, and the system will handle all the language switches.

Here's how it works in practice:

• Pattern recognition: The AI listens for sounds, rhythms, and word patterns unique to each language
• Real-time switching: When the language changes, the system adjusts immediately
• No manual setup: You don't need to specify which languages are in your audio beforehand

Speaker diarization across languages

Through speaker diarization, the system can track who's speaking even when the same person switches languages. This means if Maria speaks both Spanish and English during a meeting, the transcript will correctly show that both language segments came from Maria, not two different speakers.

This works because your voice has consistent characteristics—like pitch and speaking rhythm—that don't change when you switch languages.

Key technologies for multilingual speech-to-text

Several technologies work together to make multilingual transcription possible. Research into multilingual conversational speech recognition continues to push accuracy forward, and understanding these technologies helps you choose the right solution for your needs.

Speech-to-text API capabilities

A speech-to-text API is the core technology that converts your audio into written words. This means you send your audio file to the service, and it sends back a text document with everything that was spoken.

The best APIs handle multiple languages simultaneously, work with common formats like MP3, WAV, and M4A, and process audio in real-time or from uploaded recordings.

Language detection API functionality

Language detection is enabled with the language_detection=True parameter on the /v2/transcript endpoint—it's not a separate API product. The system automatically identifies which language is being spoken at any moment in your audio, so you don't need to manually tag sections as "Spanish," "French," or "German."

Detection happens continuously. When speakers switch languages mid-conversation, the transcription adjusts accordingly—essential for natural conversations where people code-switch between languages. For best results with Universal-3 Pro, pair language_detection with the following code-switching prompt:

"prompt": "The spoken language may change throughout the audio, transcribe in the original language mix (code-switching), preserving the words in the language they are spoken."

This prompt tells the model to preserve each language as spoken rather than defaulting everything to English. Universal-3 Pro has built-in code switching for English, Spanish, Portuguese, French, German, and Italian—no additional parameters needed beyond the prompt. For broader language coverage, set speech_models to ["universal-3-pro", "universal-2"] so the system automatically routes to the best model for each detected language.

Audio to text for Spanish and other language-specific models

AssemblyAI uses multilingual models rather than a separate user-selected model per language. Universal-2 supports 99+ languages, and Universal-3 Pro is optimized for 6 languages, including Spanish, French, and German, with support for regional dialects built in.

Language-specific features include:

• Regional accents: Mexican vs. European Spanish pronunciation differences
• Cultural expressions: Idioms and slang specific to each region
• Grammar patterns: How sentences are structured in each language

Common use cases for multilingual transcription

You'll find multilingual transcription useful in several practical situations where language barriers create communication challenges.

International business meetings

Global teams use multilingual transcription to document meetings where participants speak different languages. When your Tokyo team speaks Japanese, your Berlin engineers contribute in German, and your New York executives respond in English, the transcript captures everything in the original languages.

This creates searchable meeting records where anyone can find specific discussions, regardless of which language was used. Team members can also review what was said in their preferred language without needing translation.

Spanish audio transcription for media content

Content creators transcribe Spanish audio to expand their reach and improve accessibility. With nearly 68 million people speaking a language other than English at home in the U.S.—62% of them Spanish—the audience for multilingual content is massive. Transcription helps you create subtitles for video versions, write blog posts from episode content, and make your material accessible to hearing-impaired audiences.

Media applications include:

• Subtitle creation: Automatic captions for YouTube or social media videos
• SEO optimization: Searchable text content for better discoverability
• Content repurposing: Converting podcast episodes into written articles

Foreign language transcription in customer service

Customer service teams transcribe calls in multiple languages for quality monitoring and training purposes. When customers call speaking French or Spanish—the most common non-English language in U.S. homes—the transcribed conversation helps supervisors review service quality and create training materials, even if they don't speak those languages themselves.

This enables consistent service standards across all languages your business supports.

Building multilingual voice agents

If you're building a voice agent that needs to understand and respond in more than one language, you don't want to stitch together separate speech-to-text, LLM, and TTS providers for each language. That's where AssemblyAI's Voice Agent API comes in. It's a single WebSocket connection that handles the full pipeline—speech understanding, reasoning, and voice generation—across English, Spanish, French, German, Italian, and Portuguese.

The API is built on Universal-3 Pro, which ranks #1 on the Hugging Face Open ASR Leaderboard. That accuracy matters because if the speech-to-text layer misunderstands a word, every downstream response is wrong. At $4.50/hr flat-rate pricing and roughly 1 second of end-to-end latency, it's invisible infrastructure—your users won't know it's there, and that's the point.

Under the hood, the Voice Agent API isn't just stitching together STT, LLM, and TTS—it's an orchestrated pipeline of specialized speech models working together so the agent listens, decides, and speaks the way a human would. Key capabilities include:

• Voice focus: Background noise filtering is enabled by default, so your agent stays accurate in noisy environments and around background speakers without any extra configuration
• Semantic barge-in: Back-channels like "uh-huh" don't interrupt the agent, but genuine interruptions like "wait, stop" do—just like a real conversation
• Session resumption: If a connection drops, sessions are preserved for 30 seconds so users can reconnect without losing conversation context
• Tool calling and MCP: Your agent can perform mid-conversation actions—check a database, book an appointment, look up an order—through native function calling and MCP integration
• 34+ multilingual voices across 10 languages: English, Spanish, French, German, Italian, Hindi, Mandarin, Russian, Korean, and Japanese—so your agent speaks naturally in the user's language

Here are some real-world use cases where multilingual voice agents are already making an impact:

• Multilingual customer support bots that detect a caller's language and respond naturally, without routing to a specialized queue
• Voice-powered ordering systems for restaurants, retail, and delivery services operating in multilingual markets
• Healthcare intake agents that conduct pre-visit questionnaires in a patient's native language, improving accuracy and comfort
• Internal helpdesk agents for global companies where employees speak different languages across offices

Implementation considerations

Getting multilingual transcription into production requires attention to security, compliance, and workflow integration.

Security and compliance

Multilingual content often involves sensitive business information or personal data across international borders. Ensure your transcription solution meets security requirements for all regions where you operate.

Key security considerations:

• Data encryption: Protection during transmission and storage
• Access controls: User permissions for viewing and editing transcripts
• Compliance standards: GDPR for European content, HIPAA for medical information
• Data residency: Where your transcripts are stored geographically

Integration with existing workflows

Your transcription solution should fit into your current processes. AssemblyAI integrates with partners like Recall.ai for meeting recording and telephony tools for call center workflows.

Consider how you'll store, share, and search transcripts within your organization's existing systems.

Cost optimization strategies

Multilingual transcription costs vary based on volume, languages, and accuracy requirements. Optimize spending by using AI for initial transcription and human review only for critical sections.

Budget optimization tips:

• Estimate monthly volume: Calculate your average hours of audio across languages
• Prioritize accuracy needs: Use premium services only where essential
• Consider subscription plans: Volume discounts for predictable usage patterns
• Monitor usage: Identify opportunities to optimize language model selection

Best practices for multilingual transcription

Getting high-quality results requires proper preparation and the right approach for your specific content.

Audio quality optimization

Clear audio is the foundation of accurate transcription across all languages. Background noise, echo, and overlapping speakers significantly reduce accuracy regardless of which language is being spoken.

Recording best practices:

• Use quality microphones: Dedicated mics or headsets outperform laptop built-ins
• Control your environment: Choose quiet spaces and minimize background noise
• Establish speaking protocols: Only one person should talk at a time
• Test audio levels: Ensure consistent volume without distortion

File format considerations

Different audio formats affect transcription quality and processing speed. Uncompressed formats like WAV preserve more audio detail, potentially improving accuracy. Compressed formats like MP3 are more practical for large files and online streaming.

Format recommendations:

• Maximum quality: WAV or FLAC for critical legal or medical content
• Balanced approach: High-bitrate MP3 for most business applications
• Streaming applications: AAC or Opus for real-time transcription

Language model selection

AssemblyAI's recommended approach is to use its multilingual models, such as Universal-2 or Universal-3 Pro. In many cases, language_detection=True should be enabled rather than selecting separate single-language models. Universal-3 Pro is designed to provide strong accuracy for Spanish, French, and German within one model.

Consider these factors when selecting models:

• Regional variants: European vs. Latin American Spanish models
• Industry vocabulary: Medical, legal, or technical terminology
• Code-switching support: Frequent language mixing within conversations

AI vs. human transcription for multilingual content

Your choice between AI and human transcription depends on your accuracy needs, timeline, and budget constraints.

When to use AI transcription

AI transcription works best when you need fast results and can accept some minor errors. Modern AI models handle clear audio very well and process hours of content in minutes.

Choose AI transcription for:

• High volume processing: When you have many hours of audio to transcribe regularly
• Quick turnaround: Live events or urgent projects needing immediate results
• Clear audio quality: Recordings with minimal background noise and distinct speakers
• Budget-conscious projects: AI costs significantly less than human transcription

When human transcription is necessary

Human transcribers excel with complex content that requires perfect accuracy. They understand context, research unfamiliar terms, and make judgment calls when audio quality is poor or speakers have heavy accents.

You need human transcription for:

• Legal proceedings: Court records, depositions, or contract discussions
• Medical documentation: Patient consultations or clinical notes
• Poor audio quality: Recordings with background noise or overlapping speakers
• Technical content: Specialized terminology or industry-specific language

Hybrid approaches

Many organizations combine both approaches for optimal results. AI provides the initial transcript quickly and affordably, then human editors review and correct the most important sections.

This balances speed, cost, and accuracy while ensuring quality where it matters most.

Getting started with multilingual transcription

Whether you're transcribing recorded interviews in Spanish, live customer calls in French, or uploaded media files in German, AssemblyAI gives you a clear path from raw audio to accurate text. The process is straightforward—and you don't need to specify the language upfront if you don't want to.

How to transcribe Spanish, French, or German audio

Getting started takes three steps: sign up for a free account, grab your API key, and send your audio file to the API with language_detection=True and speech_models=["universal-3-pro", "universal-2"]. AssemblyAI automatically identifies the spoken language, routes to the best model, and returns a transcript.

For pre-recorded audio, you have two model options. Universal-2 supports 99+ languages and handles diverse audio like multilingual podcast archives or global research interviews. Universal-3 Pro is optimized for 6 languages including Spanish, French, and German, with the highest accuracy plus regional dialect recognition and native code switching.

For real-time use cases—live captioning, voice agents, agent assist—Universal-Streaming Multilingual handles the same 6 languages with low latency. The right choice depends on whether your audio is pre-recorded or live, and how many languages you need to support.

Choosing the right model

AssemblyAI offers several models tuned for different multilingual scenarios. The decision comes down to three factors: how many languages you need, whether you're processing recorded or live audio, and how much accuracy matters for your use case.

If you're transcribing Spanish, French, or German audio and accuracy is your priority, Universal-3 Pro is the clear choice for pre-recorded files. It handles regional dialects and code switching natively—so a speaker who shifts between Spanish and English mid-sentence won't trip up the model.

For teams processing audio in less common languages—Mandarin, Hindi, Arabic, Japanese, or dozens of others—Universal-2 covers 99+ languages at a lower price point. And if you need real-time transcription with multilingual support, Universal-Streaming Multilingual gives you a solid balance of speed and cost.

Your next steps with multilingual transcription

Multilingual transcription is moving fast. Voice agents that speak a customer's language, real-time captioning that works across borders, and speech-to-text accuracy that keeps improving with every model release—these aren't future promises, they're shipping today. The gap between English-only and multilingual Voice AI is closing, and the teams building multilingual capabilities now will have a significant head start.

If you're ready to start transcribing Spanish, French, German, or any of 99+ other languages, sign up for a free AssemblyAI account and send your first audio file in minutes. Set language_detection=True and let the API handle the rest.

Frequently asked questions

Can I automatically detect the language spoken in an audio file?

Yes. Set language_detection=True in your API request and AssemblyAI will automatically identify the spoken language without you specifying it upfront. The system continuously analyzes audio patterns and detects language changes in real time. With Universal-3 Pro, detection covers English, Spanish, French, German, Portuguese, and Italian. For broader coverage across 99+ languages, use speech_models=["universal-3-pro", "universal-2"] so the system routes to the best model for each detected language.

Does speech recognition handle code-switching (switching languages within audio)?

Yes, modern speech recognition systems handle code-switching—where speakers alternate between languages mid-conversation or even mid-sentence. Universal-3 Pro has built-in code switching across English, Spanish, Portuguese, French, German, and Italian. For best results, include the code-switching prompt: "The spoken language may change throughout the audio, transcribe in the original language mix (code-switching), preserving the words in the language they are spoken." Universal-2 also supports code switching across 99 languages by setting code_switching=True in the language_detection_options parameter.

How accurate is transcription for non-English languages?

Spanish, French, and German transcription accuracy is comparable to English when using models trained on those languages. Universal-3 Pro is optimized for these languages and handles regional accents—like Mexican vs. European Spanish—with strong accuracy. For less common languages, Universal-2 covers 99+ languages with accuracy that continues to improve with each model release. Audio quality is the biggest variable: clear recordings with minimal background noise produce the best results regardless of language.

Should I use separate transcription requests for each language in my multilingual audio file?

No, you should upload the entire multilingual audio file as a single request. Modern transcription systems are designed to handle multiple languages within one file automatically, and separating languages manually often reduces accuracy and creates workflow complications.

How does transcription handle Spanish regional dialects like Mexican Spanish versus Argentinian Spanish?

Models like Universal-3 Pro are trained on multiple Spanish dialects and adapt to regional pronunciation, vocabulary, and accent differences. Common variants like Mexican and European Spanish perform best, with less common dialects close behind.

‍