Research map / Label-efficient and robust learning / OOD and anomaly detection
Typicalness-Aware Learning for Failure Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Typicalness-Aware Learning (TAL), which scores each sample's typicalness from feature mean/variance statistics against a historical queue and uses it to set a dynamic logit magnitude, improving failure detection and reducing overconfidence.
Problem
Deep neural networks trained with cross-entropy are overconfident, assigning high confidence to incorrect predictions, which hurts failure detection in critical systems; atypical samples with mismatched labels drive overfitting.
Approach
Measure each sample's typicalness via the Wasserstein-style distance of its channel-wise feature mean/variance to a Historical Feature Queue of correctly predicted training samples, then assign a dynamic logit magnitude T(tau) (small for typical, large for atypical) inside a LogitNorm-style loss, and use cosine (direction) as the confidence score at inference.
Key results
- On CIFAR-100 with ResNet110 under the New FD setting, TAL reaches AURC 347.72 vs. 356.88 for LogitNorm (the best baseline) and 376.42 for MSP.
- The abstract reports more than 5% AURC improvement on CIFAR-100 over the state of the art.
- On ImageNet with ResNet-50, TAL reduces AURC by 3.7 (Old FD, Textures) and 11.6 (New FD, WILDS) points relative to MSP, while keeping accuracy comparable (76.43 vs 76.13).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dynamic magnitude T(τ) rather than fixed T with cross-entropy (ablation) | TAL · CIFAR100, Old FD | AURC: 99.60 -> 94.33 | Helped |
| use dynamic magnitude T(τ) rather than fixed T with cross-entropy (ablation) | TAL · CIFAR100, New FD | AURC: 362.77 -> 351.49 | Helped |
| add cross-entropy to fixed-T training (ablation) | TAL · CIFAR100, Old FD | AURC: 108.46 -> 99.60 | Helped |
| add cross-entropy to fixed-T training (ablation) | TAL · CIFAR100, New FD | AURC: 355.62 -> 362.77 | Hurt |
| combine TAL with FMFP | TAL · CIFAR100 vs. SVHN, New FD on ResNet110 | AURC: 347.72 -> 320.73 | Helped |
| combine TAL with FMFP | TAL · CIFAR100 vs. SVHN, New FD on WRNet | AURC: 303.18 -> 278.28 | Helped |
| combine TAL with FMFP | TAL · CIFAR100 vs. SVHN, New FD on DenseNet | AURC: 325.70 -> 296.75 | Helped |
| replace mean/variance typicality with KNN distance (ablation) | TAL · CIFAR100 | AURC | Hurt |
| replace mean/variance typicality with Gaussian Mixture Models (ablation) | TAL · CIFAR100 | AURC | Hurt |
| increase Historical Feature Queue length beyond 50,000 (ablation) | TAL · CIFAR100 | failure detection performance | Hurt |
| vary queue length from 10,000 to 50,000 (ablation) | TAL · CIFAR100 | failure detection performance | No clear effect |
| moderately increase T_max (ablation) | TAL · CIFAR10 and CIFAR100 | EAURC | Helped |
Surprising findings
- OOD-detection methods such as Energy and LogitNorm perform worse than MSP on the old-setting failure detection task, attributed to disrupting the confidence ordering among in-distribution samples.
Limitations
- Simple typicalness computation and dynamic magnitude design; more advanced designs left to future work.
- The paper states code and data will be released upon publication, not yet available at submission.
- Reported gains on some ResNet50 ImageNet settings are modest, and TAL alone is not best on all metrics.
Experimental setup
Datasets: CIFAR-100 CIFAR-10 ImageNet SVHN Textures WILDS
Benchmarks: CIFAR-100 vs SVHN ImageNet vs Textures ImageNet vs WILDS
Metrics: AURC EAURC AUROC FPR95 TNR95 AUPR-Success AUPR-Error ACC
Baselines: MSP MaxLogit Cosine Energy Entropy Mahalanobis Gradnorm SIRC LogitNorm OpenMix FMFP
Builds on: LogitNorm CORES FMFP OpenMix
Closest papers by meaning
- Interpretable Failure Detection with Human-Level Concepts (AAAI 2025 · similarity 0.87)
- Simplification Is All You Need against Out-of-Distribution Overconfidence (CVPR 2025 · similarity 0.84)
- SURE: SUrvey REcipes for building reliable and robust deep networks (CVPR 2024 · similarity 0.83)
- PAGER: Accurate Failure Characterization in Deep Regression Models (ICML 2024 · similarity 0.82)
- Failures Are Fated, But Can Be Faded: Characterizing and Mitigating Unwanted Behaviors in Large-Scale Vision and Language Models (ICML 2024 · similarity 0.82)
- CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.