Research map / Label-efficient and robust learning / OOD and anomaly detection
Can We Ignore Labels in Out of Distribution Detection?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves that unlabeled (SSL/unsupervised/zero-shot) OOD detectors fail when the surrogate objective is independent of ID labels ("label blindness"), and introduces the Adjacent OOD benchmark where ID and OOD classes are split within the same dataset, showing unlabeled methods underperform a supervised MSP baseline.
Problem
Unlabeled OOD detection (self-supervised, unsupervised, zero-shot) is promoted as cheaper and as good as supervised methods, but it is unclear whether it can reliably detect OOD inputs that overlap the in-distribution data in feature space.
Approach
Information-theoretic argument that an information-bottleneck SSL representation contains no information about labels independent of its surrogate objective, so OOD detection fails under "label blindness"; plus an Adjacent OOD benchmark that randomly splits classes within one dataset to expose this failure, which conventional far/near OOD benchmarks hide.
Key results
- Supervised MSP reaches 70.8 AUROC on Faces, 69.2 on Cars, and 78.8 on Food, while SimCLR SSD gets 55.0, 52.7, and 64.4 respectively.
- RotLoss methods score around 46 to 51 AUROC across the three adjacent OOD tasks, close to random guessing.
- CLIPN ATD reaches 81.1 AUROC on Cars and 84.9 on Food, but only 44.2 on Faces, which the authors attribute to label-text misalignment with pretraining data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use supervised MSP instead of unlabeled SimCLR SSD | OOD detection · Adjacent OOD - Faces | AUROC: 55.0±4.5 -> 70.8±0.3 | Helped |
| Use supervised MSP instead of unlabeled SimCLR SSD | OOD detection · Adjacent OOD - Cars | AUROC: 52.7±0.7 -> 69.2±0.9 | Helped |
| Use supervised MSP instead of unlabeled SimCLR SSD | OOD detection · Adjacent OOD - Food | AUROC: 64.4±0.8 -> 78.8±1.2 | Helped |
| Use CLIPN ATD instead of supervised MSP | OOD detection · Adjacent OOD - Faces | AUROC: 70.8±0.3 -> 44.2±1.4 | Hurt |
| Use CLIPN ATD instead of supervised MSP | OOD detection · Adjacent OOD - Cars | AUROC: 69.2±0.9 -> 81.1±4.3 | Helped |
| Use CLIPN ATD instead of supervised MSP | OOD detection · Adjacent OOD - Food | AUROC: 78.8±1.2 -> 84.9±0.2 | Helped |
Surprising findings
- Unlabeled SimCLR and diffusion OOD detectors perform at or near random guessing on adjacent OOD, far below a simple supervised MSP baseline.
- Zero-shot CLIPN performance depends on whether label text appears in pretraining data, not on the method itself.
Limitations
- Theory covers strict label blindness only under independence assumptions; approximate case is argued via Fano's inequality rather than proven for OOD detection.
- Adjacent OOD experiments use only three datasets (faces, cars, food) with 64x64 images and a single ResNet50 setup.
- Results for CLIPN depend on pretraining data overlap and are not controlled.
Experimental setup
Datasets: ICML Facial Expressions Stanford Cars Food-101 CIFAR10 CIFAR100 CC3M
Benchmarks: Adjacent OOD (ICML Facial Expressions, Stanford Cars, Food-101) CIFAR10/CIFAR100 adjacent OOD (appendix)
Metrics: AUROC FPR95
Baselines: Supervised MSP SimCLR KNN SimCLR SSD RotLoss KNN RotLoss SSD Diffusion inpainting OOD (LPIPS and MSE) CLIPN MSP CLIPN CTW CLIPN ATD
Builds on: SimCLR Rotation loss OOD detection SSD CLIPN information bottleneck
Closest papers by meaning
- A Noisy Elephant in the Room: Is Your Out-of-Distribution Detector Robust to Label Noise? (CVPR 2024 · similarity 0.92)
- When and How Does In-Distribution Label Help Out-of-Distribution Detection? (ICML 2024 · similarity 0.90)
- How Does Unlabeled Data Provably Help Out-of-Distribution Detection? (ICLR 2024 · similarity 0.90)
- Position: Supervised Classifiers Answer the Wrong Questions for OOD Detection (ICML 2025 · similarity 0.89)
- YolOOD: Utilizing Object Detection Concepts for Multi-Label Out-of-Distribution Detection (CVPR 2024 · similarity 0.89)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.