Research map / Label-efficient and robust learning / Noisy labels
Calibrated Disambiguation for Partial Multi-label Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PML-CD, which derives the optimal curriculum weight as the calibration curve P(clean | confidence) and uses a pre-trained histogram-based calibrator plus prototype alignment regularization to disambiguate partial multi-label image data.
Problem
Partial multi-label learning (PML), where training images carry over-annotated candidate label sets containing irrelevant labels from crowd-sourcing; existing curriculum-based weighting functions are heuristic and non-adaptive.
Approach
Shows that the optimal curriculum weight equals P(label clean | predicted confidence), i.e., the calibration curve, and learns this mapping adaptively via a transferable MLP calibrator that takes the histogram of positive-label confidences as input; adds a cosine prototype-alignment loss to the encoder.
Key results
- PML-CD achieves the best mAP in 15 of 16 dataset/noise-rate conditions versus baselines, e.g., 71.70% on VOC at 80% noise and 62.10% on COCO at 80% noise.
- Calibrator-based weighting on COCO at 80% noise reaches 61.48 mAP versus 59.93 for identity weighting and 59.79 for hard thresholding, with the optimal weighting reaching 66.42.
- Prototype alignment regularization applied in the disambiguation phase improves VOC mAP at 80% noise from 66.91 to 71.70.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PML-CD instead of the strongest baseline | PML image classification · Pascal VOC 2007, 20% noise | mAP (%): 88.99 -> 88.83 | Hurt |
| Use PML-CD instead of the strongest baseline | PML image classification · Pascal VOC 2007, 40%, 60%, and 80% noise; MS-COCO; NUS-WIDE; CUB 200 | mAP (%) | Helped |
| Replace identity weighting with the pre-trained calibrator (ablation) | PML-CD · MS-COCO, 80% noise | mAP (%): 59.93 -> 61.48 | Helped |
| Replace identity weighting with optimal weighting (ablation) | PML-CD · MS-COCO, 80% noise | mAP (%): 59.93 -> 66.42 | Helped |
| Replace identity weighting with the pre-trained calibrator (ablation) | PML-CD · MS-COCO, 80% noise | ECE of weights (%): 76.78 -> 3.74 | Helped |
| Remove prototype alignment regularization (ablation) | PML-CD · Pascal VOC 2007, 80% noise | mAP (%): 71.70 -> 66.91 | Hurt |
| Apply prototype alignment regularization in warm-up as well as disambiguation (ablation) | PML-CD · Pascal VOC 2007, 80% noise | mAP (%): 71.70 -> 71.40 | No clear effect |
| Remove prototype alignment regularization (ablation) | PML-CD · MS-COCO, 80% noise | mAP (%): 62.10 -> 60.13 | Hurt |
| Apply prototype alignment regularization in warm-up as well as disambiguation (ablation) | PML-CD · MS-COCO, 80% noise | mAP (%): 62.10 -> 62.16 | No clear effect |
Surprising findings
- PML-CD is slightly inferior to some baselines under low noise on VOC (20%), which the authors attribute to the complexity of the transformer encoder.
Limitations
- Calibrator is pre-trained only on VOC with synthetic noise, relying on transfer to other datasets.
- Gains are smaller or negative at low noise rates (e.g., VOC 20%), where simpler baselines perform better.
Experimental setup
Datasets: Pascal VOC 2007 MS-COCO 2014 NUS-WIDE CUB-200
Benchmarks: Pascal VOC 2007 MS-COCO 2014 NUS-WIDE CUB-200
Metrics: mAP expected calibration error
Baselines: PML-NI PML-LRS PLAIN BCE ASL MLDecoder UNM PML-CDCR LL-R
Builds on: ML-Decoder curriculum learning memorization effect asymmetric loss
Closest papers by meaning
- Partial Multi-Label Learning with Probabilistic Graphical Disambiguation (NeurIPS 2023 · similarity 0.89)
- Distilling Reliable Knowledge for Instance-Dependent Partial Label Learning (AAAI 2024 · similarity 0.87)
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.86)
- Controller-Guided Partial Label Consistency Regularization with Unlabeled Data (AAAI 2024 · similarity 0.86)
- Conformal Prediction for Partial Label Learning (AAAI 2025 · similarity 0.85)
- What Makes Partial-Label Learning Algorithms Effective? (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.