Research map / Large language models / Language, safety and interpretability
EuroSpeech: A Multilingual Speech Corpus
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a scalable open-source pipeline that aligns national parliamentary recordings with non-verbatim transcripts via a two-stage CER-based dynamic alignment, yielding the EuroSpeech corpus (~61k hours, 22 European languages) and showing Whisper v3 Turbo fine-tuning cuts FLEURS WER by 41.8% on six low-resource languages.
Problem
Multilingual speech datasets provide too little per-language data (typically under 1k hours) for most languages, and parliamentary recordings with non-verbatim, unsegmented transcripts are hard to convert into aligned speech-text training data across many sources.
Approach
A source-agnostic pipeline combining metadata scraping, format-agnostic download/parsing, and a two-stage coarse-to-fine CER-based dynamic alignment (with global fallback) that matches ASR segments to non-verbatim parliamentary transcripts, followed by CER-tiered filtering.
Key results
- EuroSpeech contains about 61k hours of aligned speech across 22 languages, with 50.5k hours at CER < 20%, 19 languages over 1k hours and 22 over 500 hours.
- Fine-tuning Whisper v3 Turbo on EuroSpeech reduces FLEURS WER on six languages from 29.2% to 15.1% on average, a 41.8% relative improvement.
- Maltese WER drops from 72.2% to 25.9% after fine-tuning, the largest gain among the six languages.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Maltese | WER (%): 72.2 -> 25.9; 64.1% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Icelandic | WER (%): 20.0 -> 15.0; 25.0% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Lithuanian | WER (%): 25.0 -> 15.9; 36.4% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Latvian | WER (%): 19.3 -> 11.1; 42.5% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Slovenian | WER (%): 20.5 -> 13.0; 36.7% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — Estonian | WER (%): 18.4 -> 9.9; 46.1% | Helped |
| Finetune Whisper v3 Turbo on EuroSpeech | Whisper v3 Turbo · FLEURS test — average | WER (%): 29.2 -> 15.1; 41.8% | Helped |
| Add LLM-based transcript cleaning (ablation) | EuroSpeech alignment pipeline · German parliamentary session — final aligned segments | median CER (%): 12.3% to 9.7% | Helped |
Limitations
- Parliamentary speech is formal, planned and repetitive, so models may generalize poorly to conversational speech.
- Alignment quality depends on the underlying ASR model, which is weaker for low-resource languages and degraded audio.
- Coverage is limited to 22 European languages and standard language varieties.
Experimental setup
Datasets: EuroSpeech FLEURS
Benchmarks: FLEURS
Metrics: word error rate character error rate
Baselines: Whisper v3 Turbo (pre-finetuning) VoxPopuli Common Voice MSR-86k YODAS MLS FLEURS
Builds on: VAC Whisper VoxPopuli
Closest papers by meaning
- WorldSpeech: A Multilingual Speech Corpus from Around the World (NeurIPS 2026 · similarity 0.92)
- SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset (NeurIPS 2025 · similarity 0.86)
- IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS (NeurIPS 2024 · similarity 0.86)
- WAXAL: A Large-Scale Multilingual African Language Speech Corpus (NeurIPS 2026 · similarity 0.86)
- BIGOS V2 Benchmark for Polish ASR: Curated Datasets and Tools for Reproducible Evaluation (NeurIPS 2024 · similarity 0.85)
- LAMA-UT: Language Agnostic Multilingual ASR Through Orthography Unification and Language-Specific Transliteration (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.