Research map / Large language models / Language, safety and interpretability

WorldSpeech: A Multilingual Speech Corpus from Around the World

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Speech and audio language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces WorldSpeech, a 65k-hour human-transcribed multilingual speech corpus across 76 languages built from parliamentary, broadcaster, and audiobook sources, plus an iterative alignment refinement that re-aligns audio with a fine-tuned ASR.

Problem

Most of the world's languages lack large publicly available ground-truth aligned speech data; existing parliamentary alignment approaches fail to generalize beyond Europe and degrade when the initial ASR is weak on the target language.

Approach

Extend long-audio CER-based alignment (EuroSpeech-style coarse-to-fine matching) to diverse non-European public sources, and iteratively fine-tune the alignment ASR on first-pass matches so it recovers more segments from the same human transcripts.

Key results

• Fine-tuning whisper-large-v3-turbo on WorldSpeech yields an average relative WER reduction of 63.5% across 11 typologically diverse languages (e.g., Samoan 4.72 to 0.39 WER).
• Iterative alignment refinement increases aligned hours by +19.5% (Flemish) to +201.1% (Burmese) per language, with a third pass adding only about 4.3% on average.
• WorldSpeech contains 65k aligned hours across 76 languages, with 24 languages above 1k hours and 37 above 200 hours.

Surprising findings

• Zero-shot whisper-large-v3-turbo produces WER above 1.0 for several languages (e.g., Samoan 4.72), meaning more erroneous words than reference words.

Limitations

• Data biased toward parliamentary and broadcast formal speech; speakers not demographically representative and conversational speech underrepresented.
• Alignment yield bounded by off-the-shelf ASR quality, which varies by language, lowering yield for languages where Whisper or MMS perform poorly.
• Several sources (RFA, RFE/RL) have pending redistribution status.

Experimental setup

Datasets: WorldSpeech FLEURS

Benchmarks: FLEURS

Metrics: word error rate character error rate

Baselines: Common Voice FLEURS MLS VoxPopuli YODAS Whisper Data MMS-Lab SeamlessM4T

Builds on: EuroSpeech Silero VAD Iterative pseudo-labeling Whisper

Closest papers by meaning

• EuroSpeech: A Multilingual Speech Corpus (NeurIPS 2025 · similarity 0.92)
• LAMA-UT: Language Agnostic Multilingual ASR Through Orthography Unification and Language-Specific Transliteration (AAAI 2025 · similarity 0.89)
• SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset (NeurIPS 2025 · similarity 0.88)
• WAXAL: A Large-Scale Multilingual African Language Speech Corpus (NeurIPS 2026 · similarity 0.88)
• BIGOS V2 Benchmark for Polish ASR: Curated Datasets and Tools for Reproducible Evaluation (NeurIPS 2024 · similarity 0.87)
• Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.87)