Research map / Label-efficient and robust learning / Recognition and long-tail
Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RC3P, a model-agnostic class-conditional conformal prediction method that adds label-rank calibration to class-wise score thresholding, giving class-wise coverage with smaller prediction sets than CCP.
Problem
Class-conditional conformal prediction on classification tasks with many and/or imbalanced classes produces large prediction sets, and existing class-wise methods either lack model-agnostic guarantees or still yield large sets.
Approach
Augments class-wise conformal score thresholding with a per-class label-rank calibration: for each class y, only apply the class-wise threshold to test inputs where y is ranked within the top-k(y) predictions, where k(y) is chosen so the class-wise top-k error is below alpha, and adjusts the nominal error accordingly.
Key results
- On imbalanced datasets, RC3P reduces average prediction set size by 24.47% over the better of CCP and Cluster-CP across four datasets (32.63% excluding CIFAR-10) at 90% class-conditional coverage.
- On balanced datasets with pre-trained models, RC3P reduces APSS by 32.826% on average relative to the best baseline.
- CCP, Cluster-CP, and RC3P all achieve near-zero under-coverage ratio across settings, with RC3P maintaining class-wise coverage agnostic to the classifier and data distribution.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add calibrated label-rank filtering to class-wise conformal prediction | CCP · Imbalanced datasets, four-dataset average | APSS: 24.47% | Helped |
| Add calibrated label-rank filtering to class-wise conformal prediction | CCP · Balanced datasets, four-dataset average | APSS: 32.826% | Helped |
| Add calibrated label-rank filtering to class-wise conformal prediction | CCP · CIFAR-10 imbalanced settings | APSS | No clear effect |
Surprising findings
- Cluster-CP sometimes yields larger prediction sets than the class-wise CCP baseline in the imbalanced settings.
Limitations
- Reduction in set size depends on the trade-off condition sigma_y <= 1, which is verified empirically rather than guaranteed for all classifiers.
- Coverage-controlled comparison relies on tuned inflation parameter g, with empirical gaps to population quantities.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 mini-ImageNet Food-101 Places365 iNaturalist ImageNet
Benchmarks: CIFAR-10 CIFAR-100 mini-ImageNet Food-101 Places365 iNaturalist ImageNet
Metrics: average prediction set size (APSS) under coverage ratio (UCR) under coverage gap (UCG)
Baselines: CCP Cluster-CP APS RAPS HPS
Builds on: class-wise conformal prediction adaptive prediction sets regularized adaptive prediction sets cluster conformal prediction LDAM
Closest papers by meaning
- Ensuring Class-Conditional Coverage for Pathological Workflows (Student Abstract) (AAAI 2025 · similarity 0.90)
- Class Adaptive Conformal Training (NeurIPS 2026 · similarity 0.89)
- Conformity Score Averaging for Classification (ICML 2025 · similarity 0.88)
- Optimal transport-based conformal prediction (ICML 2025 · similarity 0.88)
- Hierarchical Conformal Classification (NeurIPS 2026 · similarity 0.88)
- Conformal Prediction with Learned Features (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.