AI Paper Atlas

Research map / Learning theory and algorithms / Prediction and decision losses

Knowing what You Know: valid and validated confidence sets in multiclass and multilabel prediction

Maxime Cauchois, Suyash Gupta, John C. Duchi

JMLR 2021 · Learning theory and algorithms › Prediction and decision losses · approach: Conformal prediction sets · 43 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace marginal conformal method with CQC (conformalized quantile classification)conformal multiclass classifier · ImageNet worst-slab coverageworst-slab coverage: 3--5% better worst-slab coverageImproved
replace marginal conformal method with GIQconformal multiclass classifier · ImageNet worst-slab coverageworst-slab coverage: 5--7% better worst-slab coverageImproved
use CQC instead of marginal method on easier CIFAR-10conformal multiclass classifier · CIFAR-10 worst-slab coverageworst-slab coverageImproved
use tree-structured scores (PGM-CQC) instead of direct inner/outer method CDioCmultilabel conformal sets · Multilabel simulation coveragefraction of examples covered at level 1-alpha: 60% -> 75%Improved
use tree-based methods (PGM, ArbiTree) instead of CDioCmultilabel conformal sets · Multilabel simulation worst-slab coverage (delta=.2)worst-slab coverageImproved
use tree-based PGM-CQC instead of CDioC on PASCAL VOCmultilabel conformal sets · PASCAL VOC worst-slab coverageworst-slab coverageImproved
use scoreless ArbiTree-CQC (only predicted signs) instead of score-based PGM-CQCmultilabel conformal sets · PASCAL VOCconfidence set sizeHurt
replace single inner/outer set with union of 4 inner/outer sets (m=2)PGM-CQC on PASCAL VOC · PASCAL VOCcoverage and number of configurations: size reduced by a factor up to 4Improved
use CQC instead of marginal methodconformal multiclass classifier · ImageNet average set sizeaverage confidence set sizeHurt
use CQC instead of marginal method in multiclass Gaussian mixtureconformal multiclass classifier · Multiclass simulation minority-class coverageper-class coverageImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy