Research map / Architectures and efficiency / RNNs and normalization
Deep Speech 2 : End-to-End Speech Recognition in English and Mandarin
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add sequence-wise BatchNorm to a 9-layer network with 7 recurrent layers (ablation) | Deep Speech 2 · English held-out development set | WER: 10.83 -> 9.52 | Helped |
| Add sequence-wise BatchNorm to a shallow network with 1 recurrent layer (ablation) | Deep Speech 2 · English held-out development set | WER: 13.55 -> 14.40 | Hurt |
| Sort minibatches by utterance length during the first epoch (ablation) | Deep Speech 2 · English held-out development set | WER: 11.96 -> 10.83 | Helped |
| Replace simple RNN layers with GRU layers at fixed parameter count (ablation) | Deep Speech 2 · English held-out development set, 9 layers and 7 recurrent layers | WER: 9.52 -> 8.19 | Helped |
| Use 3 layers of 2D-invariant convolution instead of 1 layer of 1D-invariant convolution (ablation) | Deep Speech 2 · Noisy Dev | WER: 19.36 -> 14.74 | Helped |
| Use bigram outputs rather than unigram outputs at stride 4 (ablation) | Deep Speech 2 · English Dev with language model | WER: 11.92 -> 9.93 | Helped |
| Add an external 5-gram language model (ablation) | Deep Speech 2, English 9-layer model · English development set | WER: 14.93 -> 9.52 | Helped |
| Increase training data from 120 hours to 12000 hours (ablation) | Deep Speech 2 · English Regular Dev | WER: 29.23 -> 8.46 | Helped |
| Increase model size from 18 million to 100 million parameters for GRU models (ablation) | Deep Speech 2 · English Regular Dev | WER: 10.59 -> 7.78 | Helped |
| Add BatchNorm to the 5-layer, 3-RNN Mandarin model (ablation) | Deep Speech 2 · Mandarin internal Test | CER: 11.85 -> 9.39 | Helped |
| Add 2D convolution and deepen the network to 9 layers with 7 recurrent layers (ablation) | Deep Speech 2 · Mandarin internal Test | CER: 9.39 -> 7.93 | Helped |
| Use sequence-wise BatchNorm rather than normalization immediately before each recurrent nonlinearity (ablation) | Deep Speech 2 recurrent layers · Optimization | optimization performance | Helped |
Closest papers by meaning
- Analyzing Hidden Representations in End-to-End Automatic Speech Recognition Systems (NeurIPS 2017 · similarity 0.87)
- Revisiting End-to-End Speech-to-Text Translation From Scratch (ICML 2022 · similarity 0.86)
- Exploring Transfer Learning For End-to-End Spoken Language Understanding (AAAI 2021 · similarity 0.86)
- FastLongSpeech: Enhancing Large Speech-Language Models for Efficient Long-Speech Processing (NeurIPS 2025 · similarity 0.84)
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.84)
- PolyVoice: Language Models for Speech to Speech Translation (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.