Research map / Large language models / Language, safety and interpretability
“Allot?” is “A Lot!” Towards Developing More Generalized Speech Recognition System for Accessible Communication
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Quantifies the WER gap between Indic-accented and native English ASR using Wav2Vec 2.0 on three small datasets, then fine-tunes it on accented speech and tests data augmentation.
Problem
Automatic speech recognition performs much worse on accented (Indic-accented) English than on native English, hurting inclusive communication for non-native speakers.
Approach
Applies fine-tuning of pretrained Wav2Vec 2.0 to small accented English corpora (YouTube NPTEL lectures, Svarah, TTS Indic), analyzes WER per dataset and gender, catalogs mispronunciation error types, and tests audio augmentation, which did not help.
Key results
- Pre-trained Wav2Vec 2.0 WER: 51.64% on YouTube, 34.31% on Svarah, 4.60% on TTS Indic; fine-tuning reduced these to 38.12%, 20.42%, and 1.41%.
- On Svarah, fine-tuning lowered male WER from 0.331 to 0.213 and female WER from 0.303 to 0.222.
- Augmentation with all four factors (A, B, C, D) worsened WER to 0.523 (male) and 0.568 (female) versus baseline fine-tuning.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune Wav2Vec 2.0 on YouTube accented speech | Wav2Vec 2.0 · YouTube | WER: 51.64% -> 38.12% | Helped |
| Fine-tune Wav2Vec 2.0 on Svarah accented speech | Wav2Vec 2.0 · Svarah | WER: 34.31% -> 20.42% | Helped |
| Fine-tune Wav2Vec 2.0 on TTS Indic speech | Wav2Vec 2.0 · TTS Indic | WER: 4.59% -> 1.41% | Helped |
| Fine-tune on Svarah speech | Wav2Vec 2.0 · Svarah male | WER: 0.331 -> 0.213 | Helped |
| Fine-tune on Svarah speech | Wav2Vec 2.0 · Svarah female | WER: 0.303 -> 0.222 | Helped |
| Add Gaussian noise, alter duration, shift audio in time, and change frequency (ablation) | Wav2Vec 2.0 fine-tuned on Svarah · Svarah male | WER: 0.213 -> 0.523 | Hurt |
| Add Gaussian noise, alter duration, shift audio in time, and change frequency (ablation) | Wav2Vec 2.0 fine-tuned on Svarah · Svarah female | WER: 0.222 -> 0.568 | Hurt |
| Add Gaussian noise and change audio frequency (ablation) | Wav2Vec 2.0 fine-tuned on Svarah · Svarah male | WER: 0.213 -> 0.353 | Hurt |
| Add Gaussian noise and change audio frequency (ablation) | Wav2Vec 2.0 fine-tuned on Svarah · Svarah female | WER: 0.222 -> 0.386 | Hurt |
| Use data augmentation for accent variation (ablation) | Wav2Vec 2.0 fine-tuned on accented speech · Accented speech datasets | WER | Hurt |
Surprising findings
- Data augmentation, including Gaussian noise and time shifts, degraded WER rather than improving it, which the authors attribute to misalignment with transcripts and small data size.
- Fine-tuning produced larger WER gains for male than female speakers on Svarah.
Limitations
- Datasets are small (under 4 hours each) and of uneven quality; YouTube transcripts are sometimes incorrect and TTS data is synthetic.
- Batch size restricted to 1 due to compute constraints; hyperparameters largely left at defaults.
- Fine-tuning is specific to one accent and costly to scale to other accents.
Experimental setup
Datasets: NPTEL YouTube lectures Svarah TTS Indic Dataset LibriSpeech
Benchmarks: NPTEL YouTube subset Svarah TTS Indic
Metrics: word error rate
Baselines: pre-trained Wav2Vec 2.0
Builds on: Wav2Vec 2.0
Closest papers by meaning
- LAMA-UT: Language Agnostic Multilingual ASR Through Orthography Unification and Language-Specific Transliteration (AAAI 2025 · similarity 0.86)
- SwitchLingua: The First Large-Scale Multilingual and Multi-Ethnic Code-Switching Dataset (NeurIPS 2025 · similarity 0.86)
- Speech Recognition Meets Large Language Model: Benchmarking, Models, and Exploration (AAAI 2025 · similarity 0.86)
- WorldSpeech: A Multilingual Speech Corpus from Around the World (NeurIPS 2026 · similarity 0.85)
- Dynamic-SUPERB Phase-2: A Collaboratively Expanding Benchmark for Measuring the Capabilities of Spoken Language Models with 180 Tasks (ICLR 2025 · similarity 0.85)
- IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.