Research map / Label-efficient and robust learning / Recognition and long-tail
Interpretable Failure Detection with Human-Level Concepts
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ORCA, which uses ordinal ranking of CLIP similarity scores over human-level concept descriptions (top-K concepts) to compute confidence for failure detection and interpret failures.
Problem
Failure (misclassification) detection in image classifiers is hampered by overconfident predictions when confidence relies only on category-level logits; existing confidence scores lack fine-grained signals and interpretability.
Approach
Replace category-level confidence with concept-level signals: rank CLIP similarity scores of per-category visual concepts, and score confidence by how many top-K concepts belong to the predicted category, optionally weighted by a logarithmic rank weighting (ORCA-B and ORCA-R).
Key results
- On ImageNet with ViT-B/32, ORCA-B achieves FPR95 of 69.19% vs 72.91% for zero-shot MSP (about 3.7 points lower).
- On EuroSAT with ViT-B/32, ORCA-R reaches AUROC 77.55 vs 73.93 for DescCLIP+MSP and FPR95 71.29 vs 77.54.
- On CIFAR-100 with ViT-B/32, ORCA-R improves AUROC to 83.40 vs 82.54 for DescCLIP+MSP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use concept-level ordinal ranking (ORCA-R) instead of DescCLIP + MSP | CLIP ViT-B/32 · EuroSAT | AUROC: +3.6% | Helped |
| Use concept-level ordinal ranking (ORCA-R) instead of DescCLIP + MSP | CLIP ViT-B/32 · EuroSAT | FPR95: 6.25% | Helped |
| Use concept-level ordinal ranking (ORCA-R) instead of DescCLIP + MSP | CLIP ViT-B/32 · EuroSAT | ACC: +1.49% | Helped |
| Use ORCA-B instead of zero-shot + MSP | CLIP ViT-B/32 · ImageNet | FPR95: 3.72% | Helped |
| Use ORCA-B instead of DescCLIP + MSP | CLIP ViT-B/32 · ImageNet | FPR95: 2.15% | Helped |
| Increase the number of concepts (ablation) | ORCA-R on CIFAR-100 · CIFAR-100 | AUROC | Helped |
| Increase the number of concepts (ablation) | ORCA-R on CIFAR-100 · CIFAR-100 | FPR95 | Helped |
| Use logarithmic rather than exponential rank weighting (ablation) | ORCA-R · EuroSAT | failure detection performance | Helped |
| Use rank-aware ORCA-R instead of counting-based ORCA-B (ablation) | ORCA on EuroSAT · EuroSAT | AUROC | Helped |
| Use concept-based methods instead of zero-shot or ensemble category signals | CLIP on natural image benchmarks · CIFAR-100 and ImageNet | AUROC | Helped |
Surprising findings
- Ensemble of prompt templates boosts zero-shot accuracy but yields lower AUROC and higher FPR on large-scale datasets, which the authors attribute to lack of signal diversity.
- ODIN consistently degrades AUROC relative to MSP across datasets.
Limitations
- Concept collections for large-category datasets come from a prior concept bank (Yang et al.), and for small datasets are hand-selected top-10 concepts, adding manual effort.
- Gains over baselines are sometimes marginal or mixed in AUROC and ACC (e.g., ORCA-B underperforms on some metrics).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K EuroSAT RESISC45
Benchmarks: CIFAR-10 CIFAR-100 ImageNet-1K EuroSAT RESISC45
Metrics: AUROC FPR@95TPR ACC
Baselines: Zero-shot CLIP + MSP Zero-shot + ODIN Zero-shot + DOCTOR Ensemble + MSP DescCLIP + MSP DescCLIP + ODIN DescCLIP + DOCTOR
Builds on: DescCLIP maximum softmax probability DOCTOR ODIN
Closest papers by meaning
- PRIME: Prioritizing Interpretability in Failure Mode Extraction (ICLR 2024 · similarity 0.88)
- Typicalness-Aware Learning for Failure Detection (NeurIPS 2024 · similarity 0.87)
- Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models (ICML 2024 · similarity 0.83)
- Towards Trustworthy Deep Learning (AAAI 2024 · similarity 0.82)
- DiG-IN: Diffusion Guidance for Investigating Networks - Uncovering Classifier Differences Neuron Visualisations and Visual Counterfactual Explanations (CVPR 2024 · similarity 0.82)
- FaCT: Faithful Concept Traces for Explaining Neural Network Decisions (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.