Research map / Architectures and efficiency / RNNs and normalization
HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HAINAN, a TDT-based transducer trained with stochastic predictor masking so one model supports autoregressive, non-autoregressive, semi-autoregressive, and Viterbi decoding.
Problem
Transducer ASR models require autoregressive inference and are slower than non-autoregressive CTC, creating an accuracy-efficiency trade-off in speech recognition.
Approach
Train a Token-and-Duration Transducer with the predictor output randomly masked (p=0.5) so the joint network works with or without the predictor; use the NAR hypothesis's timestamps to re-predict tokens in parallel with the predictor (semi-autoregressive refinement) and a DAG-based Viterbi decoder over per-frame outputs.
Key results
- NAR HAINAN average WER 7.19 on the English leaderboard versus 7.38 for CTC, at similar decoding time (41s vs 39s).
- AR HAINAN/0 reaches average WER 6.98 vs 7.13 for TDT and 7.15 for RNN-T, with decoding time around 92s vs 88s for TDT.
- On German, SAR-1 HAINAN reaches MLS WER 4.87 vs 5.92 for CTC NAR, at 29s versus 141s for AR TDT-level accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train TDT with stochastic predictor masking | HAINAN · English leaderboard datasets (AVG) | WER (%): 7.38 -> 7.19 | Helped |
| Use non-autoregressive inference instead of CTC | English ASR · English leaderboard datasets (AVG) | WER (%): 7.38 -> 7.19 | Helped |
| Add one semi-autoregressive refinement round | HAINAN NAR · German MLS test | WER (%): 5.11 -> 4.87 | Helped |
| Add one semi-autoregressive refinement round | HAINAN NAR · German VoxPopuli test | WER (%): 9.63 -> 9.47 | Helped |
| Add a second semi-autoregressive refinement round | HAINAN SAR-1 · German MLS test | WER (%): 4.87 -> 4.83 | Helped |
| Add Viterbi decoding | HAINAN NAR · English leaderboard datasets (AVG) | WER (%): 7.19 -> 7.13 | Helped |
| Exclude duration 0 (ablation) | HAINAN · English leaderboard datasets (AVG) | WER (%): 7.10 -> 6.98 | Helped |
| Exclude duration 0 (ablation) | HAINAN · German MLS test | WER (%): 4.75 -> 4.92 | Hurt |
| Set max duration to 1 rather than 8 (ablation) | HAINAN NAR · German MLS test | WER (%): 5.09 -> 7.58 | Hurt |
| Set max duration to 2 rather than 8 (ablation) | HAINAN NAR · German MLS test | WER (%): 5.09 -> 5.79 | Hurt |
| Set max duration to 4 rather than 8 (ablation) | HAINAN NAR · German MLS test | WER (%): 5.09 -> 5.06 | Helped |
| Set max duration to 1 rather than 8 (ablation) | HAINAN NAR · German VoxPopuli test | WER (%): 9.57 -> 12.46 | Hurt |
Surprising findings
- Excluding zero-duration predictions (HAINAN/0) gives the best overall results, and max-duration 4 performs slightly better than 8 on German.
- NAR HAINAN outputs can represent alternative hypotheses (e.g., 'ice cream' vs 'i scream') across different frames, which CTC cannot do.
Limitations
- SAR cannot fix subword deletions and its deletion error count increases with refinement.
- max-duration was not extensively tuned; results depend on duration configuration.
- Evaluation limited to English and German, and models are around 1.1B parameters.
Experimental setup
Datasets: LibriSpeech Mozilla Common Voice VoxPopuli Fisher People's Speech Wall Street Journal National Speech Corpus VCTK Multilingual LibriSpeech Europarl Suno AI datasets
Benchmarks: Huggingface Open ASR Leaderboard (AMI, Earnings22, Gigaspeech, LibriSpeech clean/other, SPGISpeech, TEDLIUM, VoxPopuli) Multilingual LibriSpeech German VoxPopuli German
Metrics: WER wall-clock decoding time
Baselines: RNN-T TDT CTC stateless TDT Hybrid-TDT-CTC
Builds on: Token-and-Duration Transducer RNN-T CTC NeMo FastConformer
Closest papers by meaning
- DELTA-TTS: Adapting Autoregressive Model into a Diffusion Language Model for Text-to-Speech (NeurIPS 2026 · similarity 0.85)
- MaskGCT: Zero-Shot Text-to-Speech with Masked Generative Codec Transformer (ICLR 2025 · similarity 0.84)
- ARECHO: Autoregressive Evaluation via Chain-Based Hypothesis Optimization for Speech Multi-Metric Estimation (NeurIPS 2025 · similarity 0.84)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.84)
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.84)
- DiTAR: Diffusion Transformer Autoregressive Modeling for Speech Generation (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.