Research map / Label-efficient and robust learning / OOD and anomaly detection
Unsupervised Layer-Wise Score Aggregation for Textual OOD Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes unsupervised, data-driven aggregation of per-layer OOD scores (Mahalanobis, cosine, IRW depth, isolation forest, LOF) across all encoder layers for textual OOD detection, and introduces the LOFTER multilingual benchmark with up to 150 classes.
Problem
Existing textual OOD detectors compute anomaly scores on a manually chosen layer (usually the last layer or logits), which is rarely the best layer and performs inconsistently across tasks, languages, and class counts.
Approach
Treat the trace of OOD scores across all encoder layers as features and aggregate them with a data-driven anomaly detector trained per class on reference score sets (no OOD samples), then take the min over classes, removing manual layer selection.
Key results
- Best combination (cosine similarity score aggregated with IRW depth) reaches average AUROC 0.99 and FPR 0.02 on LOFTER.
- Last-layer/logit baselines underperform most aggregation methods; average AUROC of logits 0.75 vs 0.92 for isolation-forest aggregation of Mahalanobis scores.
- Oracle best-layer selection averages AUROC 0.94, and data-driven aggregation methods match or exceed it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Aggregate cosine-based layer scores with IRW depth instead of using the last layer | Text OOD detection on LOFTER · LOFTER | AUROC: 0.92 -> 0.99 | Helped |
| Aggregate cosine-based layer scores with IRW depth instead of using the last layer | Text OOD detection on LOFTER · LOFTER | FPR: 0.22 -> 0.02 | Helped |
| Use Isolation Forest to aggregate layer-wise scores instead of Power Means | Text OOD detection on LOFTER · LOFTER | AUROC: 0.82 -> 0.92 | Helped |
| Aggregate all layer and class scores together instead of aggregating class-wise and taking the minimum (ablation) | Layer-wise score aggregation · LOFTER | AUROC: 0.92 -> 0.83 | Hurt |
| Aggregate all layer and class scores together instead of aggregating class-wise and taking the minimum (ablation) | Layer-wise score aggregation · LOFTER | FPR: 0.21 -> 0.36 | Hurt |
Surprising findings
- The last layer is rarely the best layer for textual OOD detection, unlike in computer vision where it is typically strong.
- Aggregating over all classes at once instead of per-class performs much worse, with AUROC around 0.5 to 0.8.
Limitations
- Aggregation can remove well-handled in-distribution samples and can miss OOD samples, as stated by the authors.
- Memory overhead grows with the number of layers since one trained score per layer must be stored.
Experimental setup
Datasets: SST2 IMDB 20Newsgroups TREC-10 TREC-50 Massive Banking77 RTE MNLI HINT3 CLINC GoEmotions
Benchmarks: LOFTER
Metrics: AUROC FPR at 95% TPR AUPR-IN AUPR-OUT Detection error
Baselines: Maximum Softmax Probability Energy score Last layer Logits PW (Power means aggregation) Mean/Median score aggregation
Builds on: Mahalanobis distance OOD detection Integrated Rank-Weighted depth maximum cosine similarity OOD detection Power means feature aggregation
Closest papers by meaning
- Discriminability-Driven Channel Selection for Out-of-Distribution Detection (CVPR 2024 · similarity 0.88)
- EAT: Towards Long-Tailed Out-of-Distribution Detection (AAAI 2024 · similarity 0.88)
- Is Complex Training Necessary for Long-Tailed OOD Detection? A Re-think from Feature Geometry (NeurIPS 2026 · similarity 0.88)
- The Best of Both Worlds: On the Dilemma of Out-of-distribution Detection (NeurIPS 2024 · similarity 0.88)
- Expecting The Unexpected: Towards Broad Out-Of-Distribution Detection (NeurIPS 2024 · similarity 0.88)
- Exploring Channel-Aware Typical Features for Out-of-Distribution Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.