Research map / Label-efficient and robust learning / Recognition and long-tail
Hierarchical Selective Classification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends selective classification to hierarchies, letting models predict less specific ancestor nodes under uncertainty, with new inference rules (Climbing), a conformal-style threshold algorithm, and a large empirical study over 1,115 ImageNet models.
Problem
Standard selective classification forces a binary choice between predicting a leaf class and rejecting, discarding useful coarse information; the paper targets selective prediction over class hierarchies that lets models retreat to ancestor nodes when uncertain.
Approach
Defines hierarchical selective risk, coverage (entropy-based), and hAURC; introduces the Climbing inference rule that starts at the most likely leaf and climbs to the root until the node's summed softmax confidence exceeds a threshold, plus a split-conformal-style algorithm for choosing the threshold for a target accuracy with high probability.
Key results
- Climbing inference rule achieves mean hAURC 36.51 (x1000) vs 42.27 for selective on 1,115 ImageNet1k models, a ~14.94% hierarchical gain (vs 6.92% for Max-Coverage).
- Proposed threshold algorithm yields lower target accuracy error than DARTS at every target accuracy; e.g., at 90% target, error 0.02% vs 0.73%, with coverage 0.87 vs 0.88.
- CLIP with linear probes improves hAURC by more than 40% over vanilla counterparts, with a median improvement roughly double that of the next-best regimes (pretraining, SSL).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace whole-sample rejection with hierarchical Climbing predictions | ImageNet-1k classifiers · ImageNet-1k validation | hAURC (× 1000): 42.27±0.46 -> 36.51±0.47 | Helped |
| use Climbing instead of Max-Coverage | ImageNet-1k classifiers · ImageNet-1k validation | hAURC (× 1000): 39.99±0.53 -> 36.51±0.47 | Helped |
| replace whole-sample rejection with hierarchical Climbing predictions | iNat21 classifiers · iNat21 validation | hAURC (× 1000): 13.35±0.50 -> 10.15±0.41 | Helped |
| use Climbing instead of Max-Coverage | iNat21 classifiers · iNat21 validation | hAURC (× 1000): 11.96±0.45 -> 10.15±0.41 | Helped |
| use the hierarchical selective threshold algorithm instead of DARTS | ImageNet-1k classifiers · ImageNet-1k validation, target accuracy 99% | Target accuracy error (%): 0.63±4.2e-03 -> 0.02±1.1e-04 | Helped |
| use the hierarchical selective threshold algorithm instead of DARTS | ImageNet-1k classifiers · ImageNet-1k validation, target accuracy 99% | Coverage: 0.22±2.0e-03 -> 0.40±3.2e-04 | Helped |
| use the hierarchical selective threshold algorithm instead of DARTS | iNat21 classifiers · iNat21 validation, target accuracy 99% | Target accuracy error (%): 0.49 ± 3.0e-02 -> 0.01 ± 1.3e-03 | Helped |
| use the hierarchical selective threshold algorithm instead of DARTS | iNat21 classifiers · iNat21 validation, target accuracy 99% | Coverage: 0.39 ± 2.20e-02 -> 0.687 ± 2.49e-03 | Helped |
| add temperature scaling (ablation) | Climbing on ImageNet-1k classifiers · ImageNet-1k validation | hAURC (× 1000): 39.24±0.49 -> 36.51±0.47 | Helped |
| add temperature scaling (ablation) | Max-Coverage on ImageNet-1k classifiers · ImageNet-1k validation | hAURC (× 1000): 39.99±0.53 -> 37.21±0.52 | Helped |
| add temperature scaling (ablation) | Selective on ImageNet-1k classifiers · ImageNet-1k validation | hAURC (× 1000): 42.27±0.46 -> 41.16±0.46 | Helped |
| train with the LCA-based hierarchical loss | ImageNet classifiers · ImageNet validation | hAURC improvement: 3\% to 5\% | Helped |
Surprising findings
- Zero-shot CLIP models are weaker at hierarchical selective performance than linear-probe CLIP, with a mean relative hAURC improvement of 45% from zero-shot to linear probe.
- Improvement in hAURC does not determine the rate of accuracy improvement across training regimes, suggesting HSC gains are not solely from accuracy gains.
Limitations
- Relies only on softmax response as the confidence function.
- Training regimes were not optimized for hierarchical selective classification; the explored fine-tuning gave only 3-5% hAURC improvement.
- Evaluation focuses on image classification with pretrained ImageNet1k-style models and one additional iNat21 setup with only 6 models.
Experimental setup
Datasets: ImageNet1k ImageNet21k ImageNet12k iNat21 iNat21-Mini WordNet hierarchy
Benchmarks: ImageNet1k validation iNat21-Mini validation
Metrics: hAURC hierarchical gain target accuracy error coverage ECE AUROC
Baselines: Selective (hierarchically-ignorant) inference rule Max-Coverage DARTS Conformal risk control
Builds on: selective classification DARTS temperature scaling split conformal prediction
Closest papers by meaning
- Confidence-aware Contrastive Learning for Selective Classification (ICML 2024 · similarity 0.87)
- Deep Probabilistic Supervision for Image Classification (NeurIPS 2026 · similarity 0.84)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.84)
- ProHOC: Probabilistic Hierarchical Out-of-Distribution Classification via Multi-Depth Networks (CVPR 2025 · similarity 0.84)
- Classification with Conceptual Safeguards (ICLR 2024 · similarity 0.84)
- How Deep Networks Learn Sparse and Hierarchical Data: the Sparse Random Hierarchy Model (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.