Research map / Label-efficient and robust learning / Recognition and long-tail

CR-CTC: Consistency regularization on CTC for improved speech recognition

ICLR 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Multilingual language representation pretraining · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes CR-CTC, which enforces KL consistency between CTC distributions from two differently SpecAugment-masked views of the same speech, improving CTC to transducer/CTC-AED level on LibriSpeech, Aishell-1 and GigaSpeech.

Problem

CTC is the simplest and most efficient end-to-end ASR method but lags behind transducer and CTC/AED in recognition accuracy.

Approach

Apply a bidirectional, stop-gradient KL consistency loss between CTC posteriors of two independently time-masked SpecAugment views of the same input, with increased time masking; the authors interpret it as self-distillation between sub-models, masked prediction, and peak suppression.

Key results

• CR-CTC with Zipformer-M reaches 2.1% / 4.61% WER on LibriSpeech test-clean/test-other versus 2.52% / 6.02% for vanilla CTC at the same size.
• Pruned transducer combined with CR-CTC (Zipformer-L) reports 1.88% / 3.95% on LibriSpeech test-clean/test-other, described as a new state of the art without external LM.
• CR-CTC with Zipformer-M achieves 4.02% test WER on Aishell-1 versus 4.8% for CTC, and CR-CTC Zipformer-M gets 10.72% test WER on GigaSpeech versus 11.27% for CTC.

Measured effects reported in the paper

Surprising findings

• Applying the same larger time masking to the vanilla CTC baseline hurts performance (WER rises by 0.17/0.26), so the gain depends on the consistency loss.
• Increasing frequency masking instead of time masking degrades WER, suggesting the gain comes from masked prediction rather than input diversity alone.

Limitations

• Training requires two forward passes per batch, doubling per-step compute even though epochs are halved.
• Gains are reported mainly with Zipformer encoders and icefall recipes; a Conformer check is only on LibriSpeech.
• The mechanistic explanations (self-distillation, masked prediction, peak suppression) are supported by ablations rather than formal analysis.

Experimental setup

Datasets: LibriSpeech Aishell-1 GigaSpeech

Benchmarks: LibriSpeech test-clean LibriSpeech test-other Aishell-1 dev Aishell-1 test GigaSpeech dev GigaSpeech test

Metrics: WER CER

Baselines: CTC CTC/AED pruned transducer EMA-distilled CTC SR-CTC Conformer transducer Stateformer E-Branchformer

Builds on: CTC R-Drop cosub SpecAugment Zipformer

Closest papers by meaning

• Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework (ICLR 2024 · similarity 0.91)
• Self-Distillation Regularized Connectionist Temporal Classification Loss for Text Recognition: A Simple Yet Effective Approach (AAAI 2024 · similarity 0.87)
• Uncertainty-Aware Self-Training for CTC-Based Automatic Speech Recognition (AAAI 2025 · similarity 0.86)
• Aligner-Encoders: Self-Attention Transformers Can Be Self-Transducers (NeurIPS 2024 · similarity 0.83)
• Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.82)
• HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR (ICLR 2025 · similarity 0.82)