Research map / Large language models / Reasoning and chain of thought

The Geometric Reasoner: Manifold-Informed Latent Foresight Search for Long-Context Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes TGR, a training-free inference-time method that searches over chunk-boundary latent anchors on a unit sphere, scoring them with lookahead likelihood plus soft bumpiness and uniformity penalties, with chunk-wise KV resets for bounded memory.

Problem

Generating diverse, high-coverage long chain-of-thought trajectories under test-time compute is costly in memory and often yields redundant samples; RL-based fixes are training-heavy and can collapse the trajectory distribution.

Approach

At each chunk boundary, sample K candidate unit-norm latent anchors in the tangent neighborhood of the previous anchor, score each by a short look-ahead rollout's average log-likelihood minus soft bumpiness (second-order difference of hidden states) and a hinge uniformity penalty, inject the chosen anchor via a fixed low-rank residual, and reset the KV cache each chunk.

Key results

• TGR-Latent reaches AUC 34.8 on AIME25 and 39.5 on OlympiadBench with Qwen3-8B, matching the best RL baseline (Dele-SimKO, 34.8) on AIME25 and exceeding it on OlympiadBench (38.8) while using about 7.0k vs 8.5k average tokens.
• On code, TGR-Latent achieves AUC 73.5 on HumanEval versus 73.2 for Dele-SimKO, and the highest AUC on all three code benchmarks.
• Ablation on Qwen3-8B: removing the look-ahead value term drops average AUC from 53.2 to 43.7, the largest degradation; replacing anchors with random ones drops it to 38.3.

Surprising findings

• Hard geometric feasibility filtering is reported to have acceptance falling to under 1.5% at 14B, which motivates the soft penalty design.

Limitations

• Trades training cost for inference overhead, affecting latency-sensitive serving.
• Requires access to hidden states for anchor extraction and residual injection, so closed-box APIs are not directly supported.
• Shallow rollouts may give noisy foresight estimates for near-tied candidates.

Experimental setup

Datasets: MATH500 AIME2025 OmniMath OlympiadBench HumanEval BigCodeBench LiveCodeBench

Benchmarks: AIME25 OlympiadBench OmniMath HumanEval BigCodeBench LiveCodeBench MATH500

Metrics: Pass@k AUC over log-scale Pass@k average tokens

Baselines: Power Sampling LCoT-GRPO LCoT-SimKO Dele-GRPO Dele-SimKO TGR-Token

Builds on: Power Sampling Delethink GRPO SimKO Tree of Thoughts

Closest papers by meaning

• Geometric Latent Reasoning Induces Shorter Generations in LLMs (NeurIPS 2026 · similarity 0.87)
• Efficient Reasoning via Constrained Optimization in Latent Space (NeurIPS 2026 · similarity 0.86)
• DTS: Enhancing Large Reasoning Models via Decoding Tree Sketching (ICML 2026 · similarity 0.86)
• Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.86)
• OneVL: One-Step Latent Reasoning and Planning with Vision-Language Explanation (NeurIPS 2026 · similarity 0.85)
• LongCoT: Benchmarking Long-Horizon Chain-of-Thought Reasoning (ICML 2026 · similarity 0.85)