Research map / Architectures and efficiency / RNNs and normalization
OWLS: Scaling Laws for Multilingual Speech Recognition and Translation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OWLS, a suite of 13 open, Whisper-style multilingual ASR/ST models from 0.25B to 18B parameters trained on up to 360K hours of public speech across 150 languages, and derives scaling laws and reports emergent abilities.
Problem
Scaling behavior of multilingual speech recognition and translation models is underexplored, and existing large speech models are trained on undisclosed data and are not open or reproducible.
Approach
Train a family of transparent Whisper-style joint ASR/ST encoder-decoder models with identical hyperparameters across model sizes (0.25B-18B) and data sizes (11K-360K hours), then fit power-law scaling laws for WER/BLEU and probe emergent abilities such as in-context learning, code-switching, and mondegreens.
Key results
- Average WER on the 9B model across multi-domain English ASR nearly halves from 12.1 (0.25B) to 6.3 (9B).
- Average WER on the 50 lowest-resource FLEURS languages drops from 59 to 45 when scaling from 1B to 9B parameters.
- Only the 9B and 18B models benefit from all three in-context examples on Quechua ICL, with 9B reaching 27.4 CER at k=3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Scale model size from 1B to 9B | OWLS · FLEURS, 50 lowest-resource languages | average WER: 59 to 45 | Helped |
| Scale model size from 0.25B to 9B | OWLS · Six English ASR benchmarks | average WER: 12.1 to 6.3 | Helped |
| Scale model size from 2B to 9B | OWLS · Six English ASR benchmarks | WER: 11.3% | Helped |
| Scale model size from 1B to 9B | OWLS · English to German speech translation | BLEU: 16.6 to 28.9 | Helped |
| Increase ASR data quantity without adding data diversity | OWLS 1B · FLEURS multilingual ASR | WER/CER | Mixed |
| Add 180K hours of YODAS data, scaling training data from 180K to 360K hours | OWLS 1B · FLEURS multilingual ASR | WER/CER | Helped |
| Increase ST training data quantity | OWLS 1B · English-to-German speech translation | BLEU | Hurt |
| Scale model size | OWLS · Japanese-to-English speech translation | BLEU | No clear effect |
| Scale model size from 0.25B to 9B at matched test-time compute | OWLS · LibriSpeech test-other | WER: 8.33 vs 4.52 | Helped |
| Use 3 in-context examples instead of 2 | OWLS 9B and 18B · Quechua ASR | CER: OWLS 9B: 28.1 to 27.4; OWLS 18B: 31.3 to 28.1 | Helped |
| Use more than 2 in-context examples | OWLS 4B and smaller · Quechua ASR | CER | Hurt |
| Add contextual biasing prompts (ablation) | OWLS 0.25B · LibriSpeech test-clean | WER: 4.05 -> 97.73 | Hurt |
Surprising findings
- Amount of training data per language is only moderately predictive of WER/CER (R^2 about 0.5), so a language-agnostic data scaling law could not be fit.
- Small models (0.25B) can be unable to produce intelligible speech translation output (BLEU below 5 on 9 of 15 English-to-X pairs), and parameter scaling cannot compensate for having only 1 hour of Japanese-to-English training data.
Limitations
- Hyperparameters were held fixed across all model sizes, so the scaling curves may not reflect best-tuned performance.
- Data scaling beyond the base corpus is limited; the YODAS subset is not yet publicly released, and some training data is under non-commercial licenses.
- Some emergent-ability findings rest on small evaluation sets and a single LLM judge (OPT 2.7B) plus 13 human raters.
Experimental setup
Datasets: OWSM v3.2 (180K hours) YODAS (180K hour clean subset) FLEURS LibriSpeech AMI SPGISpeech TEDLIUM VoxPopuli GigaSpeech AISHELL ReazonSpeech Common Voice
Benchmarks: FLEURS LibriSpeech AMI SPGISpeech TEDLIUM VoxPopuli GigaGpeech AISHELL ReazonSpeech Kspon Speech Common Voice zh-TW Siminchik Quechua
Metrics: WER CER BLEU R^2 perplexity MOS
Baselines: Whisper Large V3 Canary OWSM
Builds on: OWSM Whisper ESPnet Chinchilla scaling laws
Closest papers by meaning
- Scaling Inference-Efficient Language Models (ICML 2025 · similarity 0.88)
- A Hitchhiker's Guide to Scaling Law Estimation (ICML 2025 · similarity 0.87)
- Observational Scaling Laws and the Predictability of Langauge Model Performance (NeurIPS 2024 · similarity 0.87)
- Signal and Noise: A Framework for Reducing Uncertainty in Language Model Evaluation (NeurIPS 2025 · similarity 0.87)
- Scaling Laws for Robust Comparison of Open Foundation Language-Vision Models and Datasets (NeurIPS 2025 · similarity 0.87)
- Scaling Laws with Vocabulary: Larger Models Deserve Larger Vocabularies (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.