Research map / Label-efficient and robust learning / Noisy labels
Exploiting Worker Correlation for Label Aggregation in Crowdsourcing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model inter-worker correlation via subtype mixture (EBCC) instead of independent confusion matrices (iBCC) | Bayesian classifier combination · Synthetic data, 5 workers | accuracy: 78.4% -> 85.9% | Helped |
| EBCC versus majority voting on synthetic data | EBCC · Synthetic data | accuracy: 72.0% -> 85.9% | Helped |
| Increase number of mixture components M from 2 to 5 | EBCC · Synthetic data | accuracy | No clear effect |
| Increase dataset size beyond 50k items | EBCC · Synthetic data | accuracy | Hurt |
| Use MV-estimated class distribution as prior for tau (Emp.) instead of uninformative (ablation) | EBCC M=10 · web dataset | accuracy: 46.9% -> 84.5% | Helped |
| Apply EBCC Emp. M=10 instead of best prior method iBCC-MF | EBCC · 17 real datasets | mean accuracy: 83.4% -> 84.5% | Helped |
| Use 1-coin models (ZC, GLAD, CATD) rather than confusion-matrix models | truth inference baselines · 17 real datasets | mean accuracy: 82.9% -> 82.2% | Hurt |
| EBCC on very noisy datasets (web, adult, bird, face) with more parameters | EBCC Emp. M=10 · web, adult, bird, face | accuracy: 12.0%, 2.11%, 1.86%, 1.03% below best | Hurt |
| Increase number of random restarts R | EBCC · real datasets | accuracy | Helped |
| Use unbounded large M | EBCC · real datasets | accuracy | Hurt |
Closest papers by meaning
- Mixture of Experts Based Multi-Task Supervise Learning from Crowds (AAAI 2025 · similarity 0.89)
- Label Distribution Propagation-based Label Completion for Crowdsourcing (ICML 2025 · similarity 0.88)
- Efficient Online Crowdsourcing with Complex Annotations (AAAI 2024 · similarity 0.88)
- Label Aggregation for Composite Crowd Tasks by Worker Ability Constraint Satisfaction (AAAI 2025 · similarity 0.86)
- TLLC: Transfer Learning-based Label Completion for Crowdsourcing (ICML 2025 · similarity 0.86)
- Unbiased Multi-Label Learning from Crowdsourced Annotations (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.