Research map / Architectures and efficiency / RNNs and normalization
Depth through Recurrence: Towards Ultra-Efficient On-Device ASR
Chen Feng, Tianyi Xu, Yicheng Lin, Jay Zhuo, Ramchalam Kinattinkara Ramakrishnan, Zhaocong Yuan, Chenzheng Su, Xiaopeng Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes using recurrence to reuse shared parameters for depth in an ultra-efficient on-device ASR model, aiming for high representational power at low memory and compute.
Problem
Large ASR models achieve high accuracy through depth and capacity but are infeasible on resource-constrained edge devices such as watches and glasses.
Approach
Achieve representational depth through recurrence (repeatedly applying shared layers) rather than stacking distinct layers, to get high representational power with a small parameter and compute budget.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CKA-guided grouping instead of random contiguous grouping for shared blocks (ablation) | Edge-STT block-recurrent student · LibriSpeech clean train, K in 2-5 | WER | Improved |
| increase number of shared blocks K (ablation) | Edge-STT · Open ASR Leaderboard avg | avg WER: 13.40 -> 8.57 (K=2 -> K=10) | Improved |
| increase number of shared blocks K (ablation) | Edge-Parakeet · Open ASR Leaderboard avg | avg WER: 7.62 -> 6.93 (K=4 -> K=12) | Improved |
| add MoE-LoRA personalization to block-recurrent backbone | Edge-STT K=8 · VoxCeleb2 (106 speakers) | test WER: 12.19 -> 8.03 | Improved |
| increase LoRA rank from 16 to 32 then 64 (ablation) | MoE-LoRA on Edge-STT K=8 · VoxCeleb2 | test WER: 8.18 -> 8.03 -> 8.04 | No clear effect |
| increase LoRA rank to 64 (ablation) | MoE-LoRA on Edge-Parakeet K=4 · VoxCeleb2 | test WER: 6.07 -> 6.31 (rank 32 -> 64) | Hurt |
| add user-embedding-conditioned routing (MoE-LoRA) versus plain LoRA at same budget (ablation) | Edge-STT K=8 · VoxCeleb2 | avg WER: 8.69 -> 8.03 | Improved |
| add task loss to decoder KL distillation (ablation) | Edge-STT K=4 · LibriSpeech clean/other | avg WER: 8.15 -> 7.02 | Improved |
| add encoder-level distillation at group boundaries (ablation) | Edge-STT K=4 · LibriSpeech clean/other | avg WER: 7.02 -> 6.81 | Improved |
| use block recurrence (weight sharing) in place of independent layers | STT-480ms encoder · GPU scalar kernel latency | latency: 6.22 -> 5.13 ms | Improved |
| use block recurrence in place of independent layers | STT-480ms encoder · CPU latency (Threadripper, Apple M4 Pro) | speedup: 1.03-1.07x | Improved |
| compress with block recurrence (K=4) instead of RTN INT4 quantization at same memory | STT-480ms encoder · Open ASR Leaderboard avg | avg WER: 12.26 -> 10.72 | Improved |
Closest papers by meaning
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.89)
- Zipformer: A faster and better encoder for automatic speech recognition (ICLR 2024 · similarity 0.88)
- Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.88)
- REBORN: Reinforcement-Learned Boundary Segmentation with Iterative Training for Unsupervised ASR (NeurIPS 2024 · similarity 0.87)
- Self-Taught Recognizer: Toward Unsupervised Adaptation for Speech Foundation Models (NeurIPS 2024 · similarity 0.86)
- Speech Robust Bench: A Robustness Benchmark For Speech Recognition (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
