Research map / Label-efficient and robust learning / OOD and anomaly detection
Learning by Erasing: Conditional Entropy Based Transferable Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CETOOD, a transferable OOD detector that trains an image-erasing auto-encoder (UEN) on ImageNet32 and detects OOD groups via KL divergence between log-likelihood distributions of erased patches, without retraining on new ID datasets.
Problem
Group-based OOD detection methods need retraining on each new in-distribution dataset; the paper asks how OOD detection can transfer across ID datasets without retraining.
Approach
Train a deep generative auto-encoder with a center image-erasing strategy on a complex dataset (ImageNet) so that the conditional entropy of the erased patch given its surroundings yields distinct likelihood distributions per ID dataset; detect OOD groups by KL divergence between test and ID likelihood distributions (KDE).
Key results
- Without retraining, CETOOD reaches AUROC 99.2 for MNIST vs F-MNIST and 99.7 for CIFAR10 vs SVHN at group size 5, comparable to retrained baselines.
- At group size 5, CETOOD outperforms RF-GM by 22.6% AUROC on CelebA vs CIFAR10 (91.2 vs 92.0 is not a win; the stated gain is relative to RF-GM as reported in text).
- On CIFAR10 vs CIFAR100 and related semantic splits, CETOOD achieves only 50-64 AUROC, indicating failure when ID and OOD differ semantically.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase group size from 5 to 10 (ablation) | CETOOD · OOD detection across the reported ID/OOD dataset pairs | AUROC: 99.2 -> 100.0; 99.7 -> 99.3 | Mixed |
| Use center erasing instead of corner or side erasing (ablation) | CETOOD · OOD detection, especially CelebA versus CIFAR10 and CIFAR10 versus CelebA | AUROC: 64.8 -> 99.4; 59.3 -> 99.9 | Helped |
| Use center erasing instead of corner or side erasing (ablation) | CETOOD · SVHN ID, CelebA OOD | AUROC: 100.0 -> 100.0 | No clear effect |
| Use total loss instead of generation loss only (ablation) | CETOOD · SVHN ID, CIFAR10 OOD | AUROC: $62.0$ -> 100.0 | Helped |
| Use total loss instead of reconstruction loss only (ablation) | CETOOD · SVHN ID, CIFAR10 OOD | AUROC: 76.1 -> 100.0 | Helped |
| Train on increasingly complex datasets, ending with ImageNet (ablation) | CETOOD · OOD detection across the reported ID/OOD dataset pairs | AUROC: 78.0 -> 100.0 | Helped |
| Use CETOOD instead of RF-GM | OOD detection · GS 5; CelebA ID, CIFAR10 OOD | AUROC: 92.0 -> 91.2 | No clear effect |
| Use CETOOD instead of DOCR-TC-M | OOD detection · GS 5; CIFAR10 ID, SVHN OOD | AUROC: 99.0 -> 99.7 | Helped |
Surprising findings
- Feeding the real conditional entropy into the detection algorithm sometimes performs worse than the learned model, and the center erasing strategy fails on CIFAR10 vs CIFAR100 (AUROC about 64) and semantic splits.
- The method performs poorly on CIFAR10 vs CIFAR100 and animal vs non-animal splits, close to chance level.
Limitations
- Fails to separate semantically similar ID/OOD datasets such as CIFAR10 vs CIFAR100 (AUROC around 50-64).
- Performance depends on the choice of image-erasing strategy; the chosen center erasing is empirical.
- Pre-training takes 48.3 hours on a single GPU, and some results are worse than the baselines (e.g., CIFAR10 vs SVHN AUPR and FashionMNIST vs MNIST).
Experimental setup
Datasets: ImageNet32 MNIST FashionMNIST SVHN CelebA CIFAR10 CIFAR100
Benchmarks: MNIST FashionMNIST SVHN CelebA CIFAR10 CIFAR100
Metrics: AUROC AUPR FPR at 95% TPR
Baselines: Ty-test DOCR-TC-M RF-GM
Builds on: PixelCNN++ information bottleneck Ty-test DOCR-TC-M RF-GM
Closest papers by meaning
- Mysteries of the Deep: Role of Intermediate Representations in Out of Distribution Detection (NeurIPS 2025 · similarity 0.87)
- CASE: Exploiting Intra-class Compactness and Inter-class Separability of Feature Embeddings for Out-of-Distribution Detection (AAAI 2024 · similarity 0.87)
- Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.86)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.86)
- Double Descent Meets Out-of-Distribution Detection: Theoretical Insights and Empirical Analysis on the Role of Model Complexity (NeurIPS 2025 · similarity 0.86)
- Understanding Likelihood of Normalizing Flow and Image Complexity through the Lens of Out-of-Distribution Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.