Research map / Label-efficient and robust learning / Recognition and long-tail
Conformal Prediction for Partial Label Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies split conformal prediction to partial label learning (CP-PLL), using top-1 predictions within candidate sets as noisy calibration labels, and proves a coverage interval [1-α, 1-α+1/(n+1)+ε] for the noisy setting.
Problem
Partial label learning models produce uncertain predictions because noisy false-positive candidate labels corrupt training, and their accuracy alone does not reveal reliability. Standard conformal prediction cannot be applied directly because the calibration set has no clean ground-truth labels.
Approach
Treat any pre-trained PLL model as a black box, build a noisy calibration set by taking the top-ranked label within each candidate set, apply adaptive prediction-set (APS) scoring with split conformal quantiles, and prove that coverage of the true label still holds up to an extra ε term from the candidate-flipping noise.
Key results
- On CIFAR-10 with flip rate 0.1 and alpha 0.1, CP-PLL with CWE yields average set size 1.19 and with RECORDS 1.08, versus 2.96 for PRODEN.
- On CIFAR-100 with flip rate 0.01, CWE gives set size 3.00 versus 23.70 for PRODEN and 26.80 for PICO.
- The paper reports that all base models meet the 90% coverage target on all datasets after conformal calibration.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RECORDS rather than CWE as the base PLL model | CP-PLL on CIFAR-10 · CIFAR-10 (ϵ = 0.1, α = 0.1) | average set size: 1.19 -> 1.08 | Helped |
| Use CWE rather than RECORDS as the base PLL model | CP-PLL on CIFAR-100 · CIFAR-100 (ϵ = 0.01, α = 0.1) | average set size: 12.03 -> 3.00 | Helped |
| Use CWE rather than RECORDS as the base PLL model | CP-PLL on CIFAR-10-LT · CIFAR-10-LT (ρ = 50, ϵ = 0.5, α = 0.1) | average set size: 1.08 -> 1.03 | Helped |
| Use CWE rather than RECORDS as the base PLL model | CP-PLL on CIFAR-100-LT · CIFAR-100-LT (ρ = 20, ϵ = 0.1, α = 0.1) | average set size: 4.26 -> 1.00 | Helped |
| Increase the label flipping rate | CP-PLL models · CIFAR-10 | average set size: 2.96 -> 1.06 | Mixed |
| Increase the label flipping rate | CP-PLL models · CIFAR-100 | average set size: 23.70 -> 1.85 | Mixed |
| Increase the error rate α | CP-PLL models · CIFAR-10; CIFAR-100; CIFAR-10-LT; CIFAR-100-LT | average set size | Helped |
| Increase the imbalance ratio ρ | CP-PLL models · CIFAR-10-LT; CIFAR-100-LT | average set size | No clear effect |
Surprising findings
- Average set size decreases as the candidate-label flipping rate increases, because larger candidate sets make the calibration label choice noisier and the quantile smaller.
Limitations
- Studies only the naive split conformal recipe; more advanced conformal methods are not explored.
- Evaluated only on CIFAR-style image datasets with synthetic uniform flipping of candidate labels.
- Coverage bound includes an additive epsilon term that depends on the flipping rate, so guarantee is looser under heavy noise.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CIFAR-10-LT CIFAR-100-LT
Benchmarks: CIFAR-10 CIFAR-100 CIFAR-10-LT CIFAR-100-LT
Metrics: average prediction set size empirical coverage at 1-alpha
Baselines: PRODEN PICO SOLAR RECORDS CWE
Builds on: conformal prediction adaptive prediction sets PRODEN PICO SOLAR
Closest papers by meaning
- Realistic Evaluation of Deep Partial-Label Learning Algorithms (ICLR 2025 · similarity 0.91)
- Distilling Reliable Knowledge for Instance-Dependent Partial Label Learning (AAAI 2024 · similarity 0.89)
- What Makes Partial-Label Learning Algorithms Effective? (NeurIPS 2024 · similarity 0.89)
- CroSel: Cross Selection of Confident Pseudo Labels for Partial-Label Learning (CVPR 2024 · similarity 0.89)
- Adaptive Integration of Partial Label Learning and Negative Learning for Enhanced Noisy Label Learning (AAAI 2024 · similarity 0.88)
- Does Label Smoothing Help Deep Partial Label Learning? (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.