AI Paper Atlas

Research map / Label-efficient and robust learning / OOD and anomaly detection

In or Out? Fixing ImageNet Out-of-Distribution Detection Evaluation

Julian Bitterwolf, Maximilian Müller, Matthias Hein

ICML 2023 · Label-efficient and robust learning › OOD and anomaly detection · approach: Provable and feature-based OOD detection · 11 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Replace MSP with Mahalanobis distanceViT-B-384 pretrained on IN-21K · NINCOmean FPR: 51.9 -> 27.5Improved
Replace MSP with Relative MahalanobisViT-B-384 pretrained on IN-21K · NINCOmean FPR: 51.9 -> 31.2Improved
Replace MSP with KNNViT-B-384 pretrained on IN-21K · NINCOmean FPR: 51.9 -> 62.7Hurt
Replace MSP with Relative MahalanobisViT-B-384 without pretraining · NINCOmean FPR: 69.5 -> 57.1Improved
Use CLIP plus IN-12K pretraining rather than no pretrainingViT-B-384 with MSP · NINCOmean FPR: 69.5 -> 50.2Improved
Remove ID-contaminated samples from existing OOD test datasetsMSP and Mahalanobis OOD evaluation · Cleaned subsets of four popular OOD datasetsFPR at 95% TPR: 25.6% gain of MSP vs. 33.2% gain of Mahalanobis on Species; 19.6% gain vs. 26.3% gain on PlacesImproved
Use ViM instead of MSPViT pretrained on IN-21K · OOD unit-tests: whiteFPR: 0.0 -> 100.0Hurt
Use ViM instead of MSPConvNeXt without pretraining · OOD unit-tests: blackFPR: 0.0 -> 100.0Hurt
Use feature-based OOD methods, including Mahalanobis, Relative Mahalanobis, and ViM, rather than MSPOOD detectors on NINCO · NINCOmean FPRMixed
Use zero-shot CLIP methods instead of IN-1K classifiersCLIP zero-shot models · NINCOmean FPRHurt
Use OOD unit-tests alongside natural OOD dataOOD detector evaluation · OOD unit-testsfailed unit-testsMixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy