Research map / Label-efficient and robust learning / OOD and anomaly detection
Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EOE, which prompts GPT-3.5-turbo to envision outlier class labels from ID class names (visual-similarity rule) and scores OOD with a penalized CLIP softmax, achieving zero-shot far, near and fine-grained OOD detection without any OOD data.
Problem
Zero-shot OOD detection with CLIP relies only on closed-set ID labels, so it fails on hard OOD samples; the paper asks whether potential outlier class labels can be generated without access to test-time OOD data.
Approach
Use an LLM with visual-similarity prompts (far, near, fine-grained variants) to envision outlier class names from ID labels alone, then score samples with a penalized softmax that subtracts a weighted max probability over the envisioned outlier labels.
Key results
- Far OOD on four fine-grained ID datasets: average FPR95 0.21% and AUROC 99.92% vs MCM 2.68% / 99.40%, with CLIPN averaging 0.67% FPR95.
- ImageNet-1K far OOD: average FPR95 30.09% and AUROC 92.96%, vs MCM 42.77% / 90.77%; a 12.68% FPR95 gain over MCM as stated in the introduction.
- Near OOD (ImageNet-10/20): average FPR95 9.07% and AUROC 98.59%, vs MCM 11.20% / 98.29%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LLM-envisioned outlier labels and the EOE score | CLIP OOD detection · Far OOD average across four ID datasets | FPR95: 0.67 -> 0.21 | Helped |
| Add LLM-envisioned outlier labels and the EOE score | CLIP OOD detection · ImageNet-1K far OOD average | FPR95: 30.39 -> 30.09 | Helped |
| Add LLM-envisioned outlier labels and the EOE score | CLIP OOD detection · Near OOD average (ImageNet-10/20) | FPR95: 11.20 -> 9.07 | Helped |
| Add LLM-envisioned outlier labels and the EOE score | CLIP OOD detection · Fine-grained OOD average | FPR95: 56.05 -> 60.52 | Hurt |
| Add the visual-similarity constraint to LLM prompts (ablation) | EOE on ImageNet-10 vs ImageNet-20 · Near OOD | FPR95 | Helped |
| Use the EOE score instead of MSP, Energy, MaxLogit, or MAX score (ablation) | EOE on ImageNet-10 vs ImageNet-20 · Near OOD | OOD performance | Helped |
| Use LLaMA2-7B or Claude 2 instead of GPT-3.5-turbo-16k to generate outlier labels (ablation) | EOE on ImageNet-10 · Near OOD | FPR95 | Helped |
| Vary the number of generated outlier labels (ablation) | EOE across far, near, and fine-grained OOD · Multiple OOD tasks | OOD performance | No clear effect |
| Set beta to 0.25 instead of 0 for 500 outlier labels (ablation) | EOE on ImageNet-1K · Far OOD average | FPR95: 37.54 -> 30.09 | Helped |
| Set beta to 0.50 instead of 0 for 500 outlier labels (ablation) | EOE on ImageNet-1K · Far OOD average | FPR95: 37.54 -> 28.52 | Helped |
Surprising findings
- Adding actual OOD class labels to CLIP greatly improves detection, showing closed-set ID labels alone are insufficient.
- Envisioned outlier labels improve detection even when they do not match the ground-truth OOD classes.
Limitations
- Assumes the LLM understands the ID categories; fails for expert or entirely new categories.
- Requires knowing the OOD task type (far, near, fine-grained) to select the appropriate prompt.
Experimental setup
Datasets: CUB-200-2011 Stanford Cars Food-101 Oxford-IIIT Pet ImageNet-1K iNaturalist SUN Places Texture ImageNet-10 ImageNet-20
Benchmarks: CUB-200-2011 far OOD Stanford Cars far OOD Food-101 far OOD Oxford-IIIT Pet far OOD ImageNet-1K far OOD (iNaturalist, SUN, Places, Texture) ImageNet-10/ImageNet-20 near OOD fine-grained OOD splits of CUB, Stanford Cars, Food, Oxford Pet CIFAR-10/100 OpenOOD SSB-hard and NINCO
Metrics: FPR95 AUROC AUPR
Baselines: MCM CLIPN Energy MaxLogit MSP MOS Fort et al. Energy (FT)
Builds on: MCM CLIP CLIPN Energy MaxLogit
Closest papers by meaning
- ID-like Prompt Learning for Few-Shot Out-of-Distribution Detection (CVPR 2024 · similarity 0.91)
- Out-of-Distribution Detection with Prototypical Outlier Proxy (AAAI 2025 · similarity 0.89)
- Adaptive Multi-prompt Contrastive Network for Few-shot Out-of-distribution Detection (ICML 2025 · similarity 0.89)
- Rethinking the Evaluation of Out-of-Distribution Detection: A Sorites Paradox (NeurIPS 2024 · similarity 0.89)
- Local-Prompt: Extensible Local Prompts for Few-Shot Out-of-Distribution Detection (ICLR 2025 · similarity 0.89)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.