Research map / Label-efficient and robust learning / OOD and anomaly detection
Exploiting Discrepancy in Feature Statistic for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes scoring OOD by multiplying the energy score with (1 + element-mean of penultimate features), exploiting the observation that feature element averages are larger for ID data, with a ESN-distribution theoretical analysis.
Problem
Out-of-distribution detection for CNN image classifiers: separating unseen OOD inputs from in-distribution data using post-hoc scoring without retraining.
Approach
The average of penultimate-layer feature vector elements is typically larger for ID than OOD data; multiplying this element-mean term with the energy score (on top of ReAct rectification) enlarges the ID-OOD gap, with a rectified epsilon-skew-normal analysis explaining the discrepancy.
Key results
- On ImageNet1K with ResNet50, the method reaches 19.17% average FPR95 and 95.73% AUROC, beating the best baseline LINe (20.70% FPR95, 95.03% AUROC).
- On CIFAR100 with ResNet18, average FPR95 drops from 63.06% (ReAct) to 58.38%, and with WideResNet28-10 from 67.78% to 49.25%.
- With a stronger training recipe (Ours+), CIFAR10 ResNet18 reaches 96.76% average AUROC, comparable to LogitNorm, and WRN CIFAR100 reaches 91.30% AUROC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add penultimate-layer element mean to the ReAct energy score | ReAct with ResNet50 · ImageNet1K (average) | FPR95: 20.70 -> 19.17 | Helped |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with ResNet50 · ImageNet1K (average) | AUROC: 95.03 -> 95.73 | Helped |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with ResNet18 · CIFAR10 (average) | FPR95: 31.65 -> 33.20 | Hurt |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with ResNet18 · CIFAR10 (average) | AUROC: 92.26 -> 93.22 | Helped |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with WideResNet28-10 · CIFAR10 (average) | AUROC: 93.23 -> 93.81 | Hurt |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with ResNet18 · CIFAR100 (average) | FPR95: 63.06 -> 58.38 | Helped |
| Add penultimate-layer element mean to the ReAct energy score | ReAct with WideResNet28-10 · CIFAR100 (average) | FPR95: 67.78 -> 49.25 | Helped |
| Train for 600 epochs with RandAugment (ablation) | Ours with ResNet18 · CIFAR10 (average) | AUROC: 93.22 -> 96.76 | Helped |
| Train for 600 epochs with RandAugment (ablation) | Ours with ResNet18 · CIFAR100 (average) | AUROC: 88.34% to 89.72% | Helped |
| Train for 600 epochs with RandAugment (ablation) | Ours with WideResNet28-10 · CIFAR100 (average) | AUROC: 89.70% to 91.30% | Helped |
| Add block 3 element-mean discrepancy (ablation) | Ours with ResNet50 · ImageNet1K (average) | AUROC: 95.73 -> 95.72 | No clear effect |
| Add block 2 element-mean discrepancy as well as block 3 (ablation) | Ours with ResNet50 · ImageNet1K (average) | AUROC: 95.73 -> 95.69 | No clear effect |
Surprising findings
- On CIFAR10 with WideResNet28-10 the element-mean term did not yield a new state of the art, which the authors attribute to smaller ID-OOD feature-norm gaps when there are few classes.
- Adding element-mean terms from shallow blocks 1-3 does not consistently improve average OOD performance.
Limitations
- Epsilon-skew hyperparameter is fixed at -0.4 by reference to literature rather than estimated analytically, so the theoretical estimates are approximate.
- Gains are inconsistent across benchmarks; no improvement on CIFAR10 with WideResNet28-10.
- Evaluated only on image classification with CNNs; generalization to other modalities is untested.
Experimental setup
Datasets: ImageNet1K CIFAR10 CIFAR100 iNaturalist SUN Places Textures SVHN LSUN-Crop LSUN-Resize iSUN Places365
Benchmarks: ImageNet1K OOD benchmark (iNaturalist, SUN, Places, Textures) CIFAR10 OOD benchmark CIFAR100 OOD benchmark
Metrics: FPR95 AUROC
Baselines: MSP ODIN Mahalanobis Energy ViM BATS DICE ReAct FeatureNorm LINe LogitNorm CIDER
Builds on: ReAct Energy-based OOD detection ViM DICE
Closest papers by meaning
- Learning to Shape In-distribution Feature Space for Out-of-distribution Detection (NeurIPS 2024 · similarity 0.90)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.89)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.89)
- Test-Time Linear Out-of-Distribution Detection (CVPR 2024 · similarity 0.89)
- Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.89)
- CADRef: Robust Out-of-Distribution Detection via Class-Aware Decoupled Relative Feature Leveraging (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.