Research map / Label-efficient and robust learning / OOD and anomaly detection
How Out-of-Distribution Detection Learning Theory Enhances Transformer: Learnability and Reliability
Yijin Zhou, Yutang Ge, Wenyuan XIE, Linqian Zeng, Xiaowen Dong, Yu Guang Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add synthetic rounded OOD data generation during fine-tuning (ablation) | transformer on Gaussian mixture data · Gaussian mixture toy data | ID/OOD accuracy | Improved |
| add ID-OOD binary classification loss L2 without synthetic outliers (ablation) | cross-entropy-trained transformer · Gaussian mixture toy data | OOD detection | No clear effect |
| increase weight gamma of the ID-OOD loss (ablation) | GROD loss · Gaussian mixture toy data | ID accuracy vs OOD recognition: gamma 0.0, 0.5, 0.9 | Mixed |
| replace cross-entropy-only training with GROD (L2 plus synthetic soft OOD) on DINO | DINO transformer · CIFAR-10 average FPR@95 | FPR@95 (%): 5.41 -> 3.01 (vs VIM) | Improved |
| apply GROD instead of best baseline | DINO transformer · CIFAR-100 average FPR@95 | FPR@95 (%): 24.28 -> 18.85 | Improved |
| apply GROD instead of best baseline | DINO transformer · ImageNet-200 average | FPR@95 (%): 33.05 -> 30.90 | Improved |
| apply GROD to LoRA fine-tuned Llama-3.1-8B | Llama-3.1-8B · IMDB vs Yelp | AUROC (%): 57.66 -> 67.00 | Improved |
| apply GROD to LoRA fine-tuned Llama-3.1-8B | Llama-3.1-8B · CLINC150 unknown intents | AUROC (%): 93.69 -> 93.64 | No clear effect |
| fine-tune with GROD versus NPOS and CIDER on text | Llama-3.1-8B · IMDB vs Yelp | FPR@95 (%): 99.99 -> 83.28 | Improved |
Closest papers by meaning
- Exploring the Limits of Out-of-Distribution Detection (NeurIPS 2021 · similarity 0.91)
- Out-of-Distribution Detection by Leveraging Between-Layer Transformation Smoothness (ICLR 2024 · similarity 0.90)
- Mining In-distribution Attributes in Outliers for Out-of-distribution Detection (AAAI 2025 · similarity 0.89)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.89)
- Out-of-Distribution Detection with Prototypical Outlier Proxy (AAAI 2025 · similarity 0.89)
- Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
