Research map / Label-efficient and robust learning / OOD and anomaly detection

Embedding Trajectory for Out-of-Distribution Detection in Mathematical Reasoning

NeurIPS 2024 · Label-efficient and robust learning › OOD and anomaly detection · approach: Spurious-aware OOD detection · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TV Score, a training-free OOD detector for mathematical reasoning in generative LMs that measures layer-wise trajectory volatility of Mahalanobis distances of sentence embeddings, and reports gains over embedding and uncertainty baselines on MultiArith-based ID vs. MATH/GSM8K-style OOD splits.

Problem

OOD detection for generative language models performing mathematical reasoning, where static embedding-based methods fail due to "pattern collapse" in the high-density output space of math answers.

Approach

Replace static embedding distance with the dynamic embedding trajectory across layers: map each layer's sentence embedding to a Mahalanobis distance w.r.t. ID Gaussian statistics per layer, and score OOD as the average absolute change of this distance across adjacent layers (optionally with higher-order differential smoothing).

Key results

• Far-shift OOD on Llama2-7B: TV Score reaches 98.76 AUROC and 5.21 FPR95, versus best baseline Perplexity at 85.64 AUROC and 53.06 FPR95.
• Near-shift OOD on Llama2-7B: TV Score achieves 92.64 AUROC and 28.39 FPR95, while baselines fall to roughly 45-60 AUROC.
• Online detection on Llama2-7B yields about 20-point average accuracy gain over embedding baselines (94.69 vs. 73.44 far-shift for input embedding).

Measured effects reported in the paper

Surprising findings

• Chain-of-thought solution outputs did not resolve pattern collapse: output-embedding detection with CoT still showed highly variable AUROC across ID-OOD pairs.
• Input-embedding Mahalanobis AUROC hovers near 50 in nearly half of the MATH domain pairs, indicating semantic embeddings cannot separate math domains.

Limitations

• Relatively small datasets (hundreds to thousands of samples) limit statistical power; mitigated by test sampling.
• Evaluation limited to Llama2-7B and GPT2-XL; the Llama2-7B model was fine-tuned on the ID training set and results depend on that setup.
• The theoretical proof relies on strong assumptions (Gaussian output embeddings, smooth trajectory functions, independence across layers) that are approximations.

Experimental setup

Datasets: MultiArith GSM8K SVAMP AddSub SingleEq SingleOp MATH MMLU OPUS

Benchmarks: MultiArith GSM8K SVAMP AddSub SingleEq SingleOp MATH MMLU

Metrics: AUROC FPR95 Accuracy Robustness (sampling variance) Kendall tau Spearman correlation

Baselines: Maximum Softmax Probability Monte-Carlo Dropout Sequence Perplexity Input Embedding Output Embedding

Builds on: Mahalanobis distance OOD detection (lee2018simple) embedding-based OOD detection for GLMs (ren2022out) SimCSE

Closest papers by meaning

• A Measure-Theoretic Analysis of Reasoning: Structural Generalization and Approximation Limits (NeurIPS 2026 · similarity 0.84)
• Observable Propagation: Uncovering Feature Vectors in Transformers (ICML 2024 · similarity 0.84)
• DynaMath: A Dynamic Visual Benchmark for Evaluating Mathematical Reasoning Robustness of Vision Language Models (ICLR 2025 · similarity 0.84)
• Geometric Latent Reasoning Induces Shorter Generations in LLMs (NeurIPS 2026 · similarity 0.83)
• Spatial Reasoning with Denoising Models (ICML 2025 · similarity 0.83)
• Generalization or Hallucination? Understanding Out-of-Context Reasoning in Transformers (NeurIPS 2025 · similarity 0.83)