Research map / Label-efficient and robust learning / OOD and anomaly detection
Scaling for Training Time and Post-hoc Out-of-distribution Detection Enhancement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Analyzes why activation shaping (ASH) works for post-hoc OOD detection, proposes SCALE (scaling-only post-hoc enhancement that keeps ID accuracy) and ISH (training-time intermediate tensor shaping in the backward pass) to improve OOD detection on CIFAR and ImageNet.
Problem
Post-hoc activation shaping (ASH) achieves strong OOD detection but lacks an explanation of its mechanism, and its pruning step reduces ID accuracy and can hurt OOD separation; training-time OOD enhancement methods are costly.
Approach
Theoretical and empirical analysis shows pruning hurts OOD separation while scaling helps; SCALE keeps all activations and only scales them (preserving argmax and ID accuracy), and ISH applies the same sample-specific scaling only to the backward-pass intermediate tensors during brief fine-tuning.
Key results
- SCALE on ImageNet ResNet50 (OpenOOD v1.5) reaches 59.76 FPR@95 / 81.36 AUROC near-OOD and 16.53 / 96.53 far-OOD with ID accuracy unchanged at 76.18, versus ASH-S 62.03/79.63 and 16.86/96.47.
- ISH (fine-tuned 90+10 epochs) achieves 84.01 near-OOD AUROC and 96.79 far-OOD AUROC on ImageNet, the highest among compared training-time methods.
- On CIFAR-10 with DenseNet-101, SCALE lowers average FPR@95 from 15.05 (ASH-S) to 12.57 and raises AUROC from 96.61 to 97.27.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace ASH-S with SCALE scaling all activations | ResNet50 · ImageNet-1K Near-OOD | AUROC: 79.63 -> 81.36 | Helped |
| replace ASH-S with SCALE scaling all activations | ResNet50 · ImageNet-1K Far-OOD | AUROC: 96.47 -> 96.53 | Helped |
| replace ASH-S with SCALE scaling all activations | DenseNet-101 · CIFAR-10 | AUROC: 96.61 -> 97.27 | Helped |
| replace ASH-S with SCALE scaling all activations | DenseNet-101 · CIFAR-100 | AUROC: 90.02 -> 90.74 | Helped |
| apply ISH during fine-tuning rather than AugMix training | ResNet50 · ImageNet-1K Near-OOD | AUROC: 83.55 -> 84.01; 0.46 | Helped |
| apply ISH during fine-tuning rather than AugMix training | ResNet50 · ImageNet-1K Far-OOD | AUROC: 95.99 -> 96.79; 0.8 | Helped |
| add activation pruning (ablation) | OOD detection using activation shaping · OOD detection | AUROC | Hurt |
| add activation scaling (ablation) | OOD detection using activation shaping · OOD detection | AUROC | Helped |
| increase SCALE percentile from 65 through 85 (ablation) | SCALE · ImageNet-1K Near-OOD | AUROC: 79.31 -> 81.36 | Helped |
| increase SCALE percentile beyond 85 (ablation) | SCALE · ImageNet-1K Near-OOD | AUROC: 81.36 -> 80.14; 73.40 | Hurt |
| increase SCALE percentile from 65 through 85 (ablation) | SCALE · ImageNet-1K Far-OOD | AUROC: 94.43 -> 96.53 | Helped |
| increase SCALE percentile beyond 85 (ablation) | SCALE · ImageNet-1K Far-OOD | AUROC: 96.53 -> 96.20; 93.28 | Hurt |
Surprising findings
- Pruning (rectification) of low activations hurts OOD detection in ASH, while scaling alone drives the improvement.
- Fine-tuning only the backward pass with scaled intermediate tensors improves OOD detection while ID accuracy drops only slightly.
Limitations
- Theoretical analysis relies on approximations (Gaussian assumption on activations, first-order Taylor expansion, neglecting a sigma term).
- Performance depends on percentile p, which must be tuned on a validation set; performance drops beyond p=0.85.
- ISH gives slightly lower ID accuracy than some baselines (76.74 vs AugMix 77.64).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet-1K SVHN LSUN-Crop LSUN-Resize iSUN Places365 Textures SSB-hard NINCO iNaturalist
Benchmarks: OpenOOD v1.5 ImageNet-1K CIFAR-10 OOD benchmark CIFAR-100 OOD benchmark
Metrics: FPR@95 AUROC ID accuracy
Baselines: EBO MSP MLS GEN RMDS TempScale ReAct ASH-S DICE LogitNorm CIDER RegMixup
Builds on: ASH ReAct DICE energy-based OOD score OpenOOD
Closest papers by meaning
- Towards Optimal Feature-Shaping Methods for Out-of-Distribution Detection (ICLR 2024 · similarity 0.89)
- Leveraging Perturbation Robustness to Enhance Out-of-Distribution Detection (CVPR 2025 · similarity 0.88)
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection (NeurIPS 2025 · similarity 0.88)
- Enhancing the Power of OOD Detection via Sample-Aware Model Selection (CVPR 2024 · similarity 0.87)
- Detecting Out-of-Distribution Through the Lens of Neural Collapse (CVPR 2025 · similarity 0.87)
- Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.