Research map / Label-efficient and robust learning / Recognition and long-tail

Uncertainty-Aware Self-Training for CTC-Based Automatic Speech Recognition

AAAI 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Confidence calibration objectives · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UNCAST, an output-probability-based sequence-level uncertainty estimator for CTC ASR, used to reweight pseudo-labels and regularize in-training student uncertainty during Noisy Student-style self-training.

Problem

Sequence-level uncertainty estimation for CTC-based ASR discards output probability information (relying on textual edit distance over MC Dropout decodes) and ignores data uncertainty, limiting its usefulness for pseudo-label filtering in self-training.

Approach

Defines a length-normalized CTC negative log-likelihood of the decoded sequence as data uncertainty, and max over MC Dropout samples as model uncertainty; uses teacher uncertainty as soft loss attenuation for pseudo-labels and differentiable student uncertainty as an in-training regularizer.

Key results

• UNCAST reaches 9.35% WER on test-other with LS-100 plus LS-360 unlabeled, versus 9.85% for DUST and 10.25% for Noisy Student.
• With LS-500 unlabeled and a second iteration, UNCAST achieves 9.16% test-other WER versus 9.46% for DUST.
• Binned analysis shows the proposed uncertainty gives monotonic WER across bins, while MCD edit-distance gives zero uncertainty for many bins.

Measured effects reported in the paper

Surprising findings

• Noisy Student without any filtering degrades performance on the second iteration with LS-360 and LS-500 unlabeled data in some metrics.
• Using the 360-hour unlabeled set outperformed the larger 500-hour set in the first iteration, attributed to incorrect pseudo-labels on the harder train-other data.

Limitations

• Data and model uncertainty are not disentangled; the authors note this as future work.
• Evaluated only on LibriSpeech with a single WavLM Base+ backbone and CTC head.
• Requires N MC Dropout forward passes for uncertainty estimation, adding inference cost.

Experimental setup

Datasets: LibriSpeech train-clean-100 LibriSpeech train-clean-360 LibriSpeech train-other-500

Benchmarks: LibriSpeech dev-clean LibriSpeech dev-other LibriSpeech test-clean LibriSpeech test-other

Metrics: word error rate

Baselines: Noisy Student DUST Seed model Oracle model

Builds on: Noisy Student Monte Carlo Dropout Vyas et al. 2019 MCD edit-distance uncertainty DUST Kendall and Gal 2017 uncertainty

Closest papers by meaning

• CR-CTC: Consistency regularization on CTC for improved speech recognition (ICLR 2025 · similarity 0.86)
• Align With Purpose: Optimize Desired Properties in CTC Models with a General Plug-and-Play Framework (ICLR 2024 · similarity 0.84)
• Speech Robust Bench: A Robustness Benchmark For Speech Recognition (ICLR 2025 · similarity 0.84)
• Unified Speech Recognition: A Single Model for Auditory, Visual, and Audiovisual Inputs (NeurIPS 2024 · similarity 0.84)
• What Was That Again? Certified Robustness for Automatic Speech Recognition (NeurIPS 2026 · similarity 0.84)
• HAINAN: Fast and Accurate Transducer for Hybrid-Autoregressive ASR (ICLR 2025 · similarity 0.83)