Research map / Learning theory and algorithms / Prediction and decision losses
Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction
Maxime Cauchois, Suyash Gupta, John C. Duchi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace marginal conformal method with CQC (conformalized quantile classification) | conformal multiclass classifier · ImageNet worst-slab coverage | worst-slab coverage: 3--5% better worst-slab coverage | Improved |
| replace marginal conformal method with GIQ | conformal multiclass classifier · ImageNet worst-slab coverage | worst-slab coverage: 5--7% better worst-slab coverage | Improved |
| use CQC instead of marginal method on easier CIFAR-10 | conformal multiclass classifier · CIFAR-10 worst-slab coverage | worst-slab coverage | Improved |
| use tree-structured scores (PGM-CQC) instead of direct inner/outer method CDioC | multilabel conformal sets · Multilabel simulation coverage | fraction of examples covered at level 1-alpha: 60% -> 75% | Improved |
| use tree-based methods (PGM, ArbiTree) instead of CDioC | multilabel conformal sets · Multilabel simulation worst-slab coverage (delta=.2) | worst-slab coverage | Improved |
| use tree-based PGM-CQC instead of CDioC on PASCAL VOC | multilabel conformal sets · PASCAL VOC worst-slab coverage | worst-slab coverage | Improved |
| use scoreless ArbiTree-CQC (only predicted signs) instead of score-based PGM-CQC | multilabel conformal sets · PASCAL VOC | confidence set size | Hurt |
| replace single inner/outer set with union of 4 inner/outer sets (m=2) | PGM-CQC on PASCAL VOC · PASCAL VOC | coverage and number of configurations: size reduced by a factor up to 4 | Improved |
| use CQC instead of marginal method | conformal multiclass classifier · ImageNet average set size | average confidence set size | Hurt |
| use CQC instead of marginal method in multiclass Gaussian mixture | conformal multiclass classifier · Multiclass simulation minority-class coverage | per-class coverage | Improved |
Closest papers by meaning
- Conformal Prediction for Class-wise Coverage via Augmented Label Rank Calibration (NeurIPS 2024 · similarity 0.89)
- Conformal Prediction for Partial Label Learning (AAAI 2025 · similarity 0.88)
- Rectifying Conformity Scores for Better Conditional Coverage (ICML 2025 · similarity 0.88)
- Conformal Structured Prediction (ICLR 2025 · similarity 0.88)
- Conformity Score Averaging for Classification (ICML 2025 · similarity 0.87)
- Conformal Prediction with Learned Features (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
