Research map / Label-efficient and robust learning / Recognition and long-tail
Who Said What: Modeling Individual Labelers Improves Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model each annotator separately instead of training on average opinions | BN to DN · Diabetic retinopathy test set | five-class Error (%): 23.83 -> 21.86 | Helped |
| Learn shared weights to combine doctor predictions | DN to WDN · Diabetic retinopathy test set | five-class Error (%): 21.86 -> 20.58 | Helped |
| Make doctor averaging weights image-specific (ablation) | WDN to IWDN · Diabetic retinopathy test set | five-class Error (%): 20.58 -> 20.63 | Hurt |
| Add a size-3 bottleneck to image-specific averaging weights (ablation) | IWDN to BIWDN · Diabetic retinopathy test set | five-class Error (%): 20.63 -> 20.83 | Hurt |
| Train BN with five-class rather than binary loss (ablation) | BN · Diabetic retinopathy test set | Binary AUC (%): 95.58 -> 97.11 | Helped |
| Use EM-updated labels to train BN | BN · Diabetic retinopathy test set | five-class Error (%): 23.83 -> 23.74 | No clear effect |
| Apply the multi-class noise-model loss to DN | DN · Diabetic retinopathy test set | five-class Error (%): 21.86 -> 22.76 | Hurt |
| Use mean class balancing (ablation) | DR classification training · Diabetic retinopathy test set | classification performance | Hurt |
| Use averaged doctor-model predictions as the averaging-logit training target (ablation) | WDN averaging logits · Diabetic retinopathy test set | classification performance | Hurt |
| Use the alternative symmetric, doctor-specific noise model (ablation) | DN noise modeling · Diabetic retinopathy test set | classification performance | Hurt |
Closest papers by meaning
- Bayesian Inference for Correlated Human Experts and Classifiers (ICML 2025 · similarity 0.85)
- From Biased Selective Labels to Pseudo-Labels: An Expectation-Maximization Framework for Learning from Biased Decisions (ICML 2024 · similarity 0.82)
- Coherent Hierarchical Multi-Label Learning to Defer for Medical Imaging (NeurIPS 2026 · similarity 0.82)
- Defining Expertise: Applications to Treatment Effect Estimation (ICLR 2024 · similarity 0.82)
- Domain Adaptive Diabetic Retinopathy Grading with Model Absence and Flowing Data (CVPR 2025 · similarity 0.82)
- FOCUS: Benchmarking Retinal Model Generalization from Foundation Vision Encoders to Multimodal LLMs (NeurIPS 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.