Research map / Large language models / Language, safety and interpretability
IndicVoices-R: Unlocking a Massive Multilingual Multi-speaker Speech Corpus for Scaling Indian TTS
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds IndicVoices-R, a 1,704-hour multi-speaker Indian TTS dataset by denoising, dereverberating and enhancing the IndicVoices ASR corpus, plus a zero-shot/few-shot/many-shot speaker benchmark and a VoiceCraft TTS fine-tuned on it.
Problem
Indian languages lack large, high-quality, multi-speaker TTS data; existing ASR corpora are noisy, low-quality recordings, limiting zero-shot speaker generalization in Indian TTS.
Approach
Repurpose the IndicVoices ASR corpus for TTS through a restoration pipeline (HTDemucs demixing, VoiceFixer dereverberation, DeepFilterNet3 enhancement) with acoustic-metric filtering, using English-trained enhancement models that generalize cross-lingually to Indian speech.
Key results
- IV-R provides 1,704 hours from 10,496 speakers across 22 Indian languages, with SNR (60.47 dB) and C50 (53.45 dB) comparable to gold-standard TTS datasets.
- VoiceCraft fine-tuned on IV-R achieves zero-shot speaker similarity of 88.18 vs 78.93 for the IndicTTS-fine-tuned model, with slightly lower NORESQA (3.64 vs 3.83).
- IV-R's N-MOS (3.38) is below LJSpeech (4.36) and IndicTTS (4.29), indicating lower perceived quality than studio datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune VoiceCraft on IndicVoices-R instead of IndicTTS | VoiceCraft 830M pretrained on GigaSpeech · IndicVoices-R Benchmark zero-shot | S-SIM: 78.93 -> 88.18 | Helped |
| Fine-tune VoiceCraft on IndicVoices-R instead of IndicTTS | VoiceCraft 830M pretrained on GigaSpeech · IndicVoices-R Benchmark zero-shot | NORESQA: 3.83 -> 3.64 | Hurt |
Surprising findings
- IV-R has lower NORESQA-MOS than the studio-quality datasets despite comparable SNR and C50, and the zero-shot NORESQA score of the IV-R model is slightly below the IndicTTS model.
Limitations
- Conversational subset excluded due to 8 kHz sampling rate, so only read and extempore 44.1 kHz audio is used.
- Dataset does not reach studio-recording quality; N-MOS is lower than studio TTS datasets.
- Uses graphemes rather than phonemes because phonemizers are unavailable for all Indian languages.
Experimental setup
Datasets: IndicVoices IndicTTS LibriTTS LJSpeech GigaSpeech
Benchmarks: IndicVoices-R Benchmark
Metrics: NORESQA-MOS SNR C50 F0 speaker similarity (cosine, S-SIM)
Baselines: IndicTTS LJSpeech LibriTTS Google-CS LIMMITS
Builds on: VoiceCraft LibriTTS-R HTDemucs VoiceFixer DeepFilterNet3
Closest papers by meaning
- NaturalSpeech 2: Latent Diffusion Models are Natural and Zero-Shot Speech and Singing Synthesizers (ICLR 2024 · similarity 0.88)
- VAANI: Capturing the language landscape for an inclusive digital India (NeurIPS 2026 · similarity 0.87)
- Scaling Speech-Text Pre-training with Synthetic Interleaved Data (ICLR 2025 · similarity 0.87)
- WorldSpeech: A Multilingual Speech Corpus from Around the World (NeurIPS 2026 · similarity 0.86)
- WAXAL: A Large-Scale Multilingual African Language Speech Corpus (NeurIPS 2026 · similarity 0.86)
- Enhancing Zero-Shot Multi-Speaker TTS with Negated Speaker Representations (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.