Research map / Large language models / Reasoning and chain of thought

Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes training-free Latent-Trajectory signals computed from segment-averaged hidden states of reasoning traces to predict answer correctness and guide answer selection and early path selection, reducing token usage versus majority voting.

Problem

Identifying which reasoning traces of reasoning LLMs will yield correct answers, so that inference-time scaling (multi-sample aggregation) can be made cheaper and more accurate without costly verifiers or annotations.

Approach

Three training-free signals over the temporal trajectory of layer-averaged segment hidden states: Net Change (norm of start-to-end drift), Cumulative Change (summed segment-to-segment update norms), and Aligned Change (cosine alignment of intermediate updates with the overall drift), used both for prediction and for early stopping in sequential sampling.

Key results

• Latent-Trajectory signals achieve mean ROC-AUC about 0.71-0.74 across datasets, beating cross-layer (0.58-0.67) and output-distribution baselines (entropy 0.44, perplexity 0.49).
• LT-based answer selection cuts token usage by about 48% on average versus MV@5 while improving accuracy by 2.64% on average (up to 70% token reduction).
• Early path selection at 2k tokens with a random forest achieves average +2.1% accuracy and 61% token reduction relative to MV@5.

Surprising findings

• Cumulative Change is negatively correlated with accuracy, so longer total representational travel signals incorrect traces.
• Output-distribution measures such as entropy and perplexity perform near or below chance for these reasoning models.

Limitations

• Evaluated only on three 14B open reasoning models and three domains.
• Threshold calibration and combined score use simple lightweight heuristics; learned classifiers not explored.
• Accuracy gains are small on some settings (e.g., -1.4% for Shortest@5 average drop and some negative LT cases), and variance across models is large.

Experimental setup

Datasets: GPQA Diamond AIME 2025 TSP (stratified subsample n=180)

Benchmarks: GPQA Diamond AIME 2025 TSP

Metrics: ROC-AUC Spearman correlation accuracy average samples token savings

Baselines: Majority Vote@5 Shortest@5 Cross-Layer Magnitude Cross-Layer Angle Logit Margin Entropy Perplexity

Builds on: self-consistency majority voting

Closest papers by meaning

• Probing the Trajectories of Reasoning Traces in Large Language Models (NeurIPS 2026 · similarity 0.93)
• The Illusion of Thinking: Understanding the Strengths and Limitations of Reasoning Models via the Lens of Problem Complexity (NeurIPS 2025 · similarity 0.90)
• Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens (NeurIPS 2026 · similarity 0.90)
• On the Bias of Next-Token Predictors Toward Systematically Inefficient Reasoning: A Shortest-Path Case Study (NeurIPS 2025 · similarity 0.90)
• Reasoning-Trace Collapse: Evaluating the Loss of Explicit Reasoning During Fine-Tuning (NeurIPS 2026 · similarity 0.90)
• Demystifying Reasoning Dynamics with Mutual Information: Thinking Tokens are Information Peaks in LLM Reasoning (NeurIPS 2025 · similarity 0.90)