AI Paper Atlas

Research map / Label-efficient and robust learning / Recognition and long-tail

CrowdSpeech and Vox DIY: Benchmark Dataset for Crowdsourced Audio Transcription

Nikita Pavlichenko, Ivan Stelmakh, Dmitry Ustalov

NeurIPS 2021 · Label-efficient and robust learning › Recognition and long-tail · approach: Dense features and foundation priors · 6 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace ROVER with fine-tuned T5 (ST) aggregationaggregation method · CrowdSpeech test-cleanWER (%): 7.29 -> 6.21Improved
replace ROVER with T5 (ST) aggregationaggregation method · CrowdSpeech test-otherWER (%): 13.41 -> 11.80Improved
further fine-tune T5 (ST) on CrowdSpeech dev sets (ST+FT) (ablation)T5 (ST) · CrowdSpeech test-cleanWER (%): 6.21 -> 5.32Improved
further fine-tune T5 (ST) on CrowdSpeech dev sets (ST+FT) (ablation)T5 (ST) · CrowdSpeech test-otherWER (%): 11.80 -> 10.46Improved
fine-tune original T5 on large train-clean instead of ST+FT (ablation)T5 · CrowdSpeech test-cleanWER (%): 5.32 -> 5.22Improved
fine-tune on train-clean only (T5 FT) instead of ST+FT (ablation)T5 · CrowdSpeech test-otherWER (%): 10.46 -> 11.67Hurt
use RASA instead of ROVERaggregation baseline · CrowdSpeech test-cleanWER (%): 7.29 -> 8.60Hurt
use HRRASA instead of ROVERaggregation baseline · CrowdSpeech test-otherWER (%): 13.41 -> 15.66Hurt
use HRRASA instead of RASAaggregation baseline · CrowdWSA2019 J1WER (%): 65.86 -> 67.57Hurt
use RASA/HRRASA instead of ROVER on translation dataaggregation baseline · CrowdWSA2019 T1WER (%): 51.35 -> 48.29Improved
use non-trivial baselines (ROVER) instead of Random on low-noise dataaggregation baseline · VoxDIY ruWER (%): 7.09 -> 1.92Improved
allow aggregation to modify transcriptions (ROVER vs Oracle selection gap)ROVER · CrowdSpeech test-otherWER (%): 13.41 -> 8.50Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy