Research map / Label-efficient and robust learning / Recognition and long-tail
CrowdSpeech and Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription
Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ROVER with fine-tuned T5 (ST) aggregation | aggregation method · CrowdSpeech test-clean | WER (%): 7.29 -> 6.21 | Improved |
| replace ROVER with T5 (ST) aggregation | aggregation method · CrowdSpeech test-other | WER (%): 13.41 -> 11.80 | Improved |
| further fine-tune T5 (ST) on CrowdSpeech dev sets (ST+FT) (ablation) | T5 (ST) · CrowdSpeech test-clean | WER (%): 6.21 -> 5.32 | Improved |
| further fine-tune T5 (ST) on CrowdSpeech dev sets (ST+FT) (ablation) | T5 (ST) · CrowdSpeech test-other | WER (%): 11.80 -> 10.46 | Improved |
| fine-tune original T5 on large train-clean instead of ST+FT (ablation) | T5 · CrowdSpeech test-clean | WER (%): 5.32 -> 5.22 | Improved |
| fine-tune on train-clean only (T5 FT) instead of ST+FT (ablation) | T5 · CrowdSpeech test-other | WER (%): 10.46 -> 11.67 | Hurt |
| use RASA instead of ROVER | aggregation baseline · CrowdSpeech test-clean | WER (%): 7.29 -> 8.60 | Hurt |
| use HRRASA instead of ROVER | aggregation baseline · CrowdSpeech test-other | WER (%): 13.41 -> 15.66 | Hurt |
| use HRRASA instead of RASA | aggregation baseline · CrowdWSA2019 J1 | WER (%): 65.86 -> 67.57 | Hurt |
| use RASA/HRRASA instead of ROVER on translation data | aggregation baseline · CrowdWSA2019 T1 | WER (%): 51.35 -> 48.29 | Improved |
| use non-trivial baselines (ROVER) instead of Random on low-noise data | aggregation baseline · VoxDIY ru | WER (%): 7.09 -> 1.92 | Improved |
| allow aggregation to modify transcriptions (ROVER vs Oracle selection gap) | ROVER · CrowdSpeech test-other | WER (%): 13.41 -> 8.50 | Mixed |
Closest papers by meaning
- Synthio: Augmenting Small-Scale Audio Classification Datasets with Synthetic Data (ICLR 2025 · similarity 0.85)
- WorldSpeech: A Multilingual Speech Corpus from Around the World (NeurIPS 2026 · similarity 0.85)
- Massive Sound Embedding Benchmark (MSEB) (NeurIPS 2025 · similarity 0.84)
- EuroSpeech: A Multilingual Speech Corpus (NeurIPS 2025 · similarity 0.84)
- Bridging the Data Provenance Gap Across Text, Speech, and Video (ICLR 2025 · similarity 0.83)
- Coupled Confusion Correction: Learning from Crowds with Sparse Annotations (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
