Research map / Learning theory and algorithms / Prediction and decision losses

Scaling Full Conformal Image Classifiers

NeurIPS 2026 · Learning theory and algorithms › Prediction and decision losses · approach: Conformal prediction sets · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes T-FCP, which prunes labels with a zero-shot CLIP-based inductive conformal predictor and runs full conformal prediction only on surviving candidates, plus SO-LDA, a rank-one-update LDA solver, to make full conformal image classification feasible on ImageNet-scale label spaces.

Problem

Full conformal prediction is data-efficient but computationally prohibitive for image classification with large label spaces, because it requires candidate-specific model refits at test time; split conformal prediction wastes calibration data and yields unstable finite-sample coverage.

Approach

Combine an ICP pruning stage (using zero-shot CLIP prototypes, error budget alpha_ICP) with FCP on the survivors via intersection of prediction sets (union-bound guarantee, alpha = alpha_ICP + alpha_FCP), and make FCP cheap with SO-LDA, a LDA solver using zero-shot-anchored residual centering and Sherman-Morrison rank-one inverse-covariance updates.

Key results

• T-FCP with SO-LDA achieves 90.7% average coverage at alpha=0.10 with 81.8% valid runs, versus 77.1% for SCP, while reducing median set size from 2.1 (SCP) to 1.9.
• On ImageNet, T-FCP reduces per-image latency to about 30 ms, versus about 0.33 s for FCP with SO-LDA and about 500 s for FCP with gradient descent.
• SO-LDA approaches the gradient-descent linear probe accuracy within about 1.5% (e.g., 80.3% vs 79.9% for SCP with GD on average across 11 datasets, reported in the main table).

Surprising findings

• SCP with gradient descent achieves similar mean accuracy to FCP with SO-LDA, yet FCP yields smaller median sets and more singletons; FCP with SO-LDA produces longer tails in set size.
• Under low calibration data (N = C x 4), full conformal procedures show slight under-coverage, and the exact non-online FCP shows the same instability.

Limitations

• SO-LDA remains less accurate than iterative gradient-based linear probes, so SCP with GD may yield more efficient sets in some cases.
• The diagonal-loading approximation means T-FCP with SO-LDA lacks a strict formal guarantee, though ablations show negligible differences.
• The union-bound allocation makes T-FCP sets slightly more conservative than FCP.

Experimental setup

Datasets: ImageNet SUN397 FGVCAircraft EuroSAT StanfordCars Food101 OxfordPets Flowers102 Caltech101 DTD UCF101 ImageNet-V2

Benchmarks: ImageNet SUN397 FGVCAircraft EuroSAT StanfordCars Food101 OxfordPets Flowers102 Caltech101 DTD UCF101 ImageNet-V2

Metrics: empirical coverage coverage standard deviation (2 sigma) %Valid mean set size median set size % singletons accuracy latency

Baselines: ICP (zero-shot) SCP with gradient descent linear probe ICP-T with Conf-OT ICP-T with TIM SS-Text vanilla LDA

Builds on: full conformal prediction inductive conformal prediction CLIP Simple-shot Conf-OT

Closest papers by meaning

• Conformal Prediction for Zero-Shot Models (CVPR 2025 · similarity 0.90)
• Singleton-Optimized Conformal Prediction (ICLR 2026 · similarity 0.88)
• Hierarchical Conformal Classification (NeurIPS 2026 · similarity 0.85)
• Class Adaptive Conformal Training (NeurIPS 2026 · similarity 0.85)
• Conformal Structured Prediction (ICLR 2025 · similarity 0.84)
• On Temperature Scaling and Conformal Prediction of Deep Classifiers (ICML 2025 · similarity 0.84)