Research map / Large language models / Reasoning and chain of thought

Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Efficient and adaptive thinking · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LTPO, a parameter-free test-time method that optimizes latent thought vectors per instance with a policy-gradient loop driven by a confidence reward from the frozen LLM's own output distributions.

Problem

Latent reasoning methods with offline-trained projection modules are brittle on hard, out-of-distribution reasoning tasks (e.g., AIME), collapsing to near-zero accuracy.

Approach

Treat K latent thought vectors inserted into the prompt as policy parameters and refine them at test time via REINFORCE with a dense intrinsic reward (mean negative log-probability of top-k tokens at thought positions), with no weight updates and no text decoding inside the loop.

Key results

• On Qwen-2.5-7B-Instruct, LTPO reaches 16.67% on AIME2024 and 13.33% on AIME2025 versus 10.00% for Zero-Shot CoT and 0.00% for SoftCoT on AIME2025.
• Average accuracy across five math benchmarks with LLaMA-3.1-8B-Instruct is 48.66% for LTPO versus 42.35% for Zero-Shot CoT and 45.02% for the best LatentSeek variant.
• Average inference time on GSM8K/AIME2024/AIME2025 is 20.79 s for LTPO versus 41.33 s for Zero-Shot CoT and over 500 s for LatentSeek; with Qwen-3-14B at 64k tokens, LTPO averages 80.00% vs 75.00% for Zero-Shot CoT on AIME.

Surprising findings

• SoftCoT scores 0.00% on AIME2024 and AIME2025 across all tested models, despite being competitive on simpler benchmarks.
• The confidence reward can be maximized on a fluent but incorrect reasoning path, so higher confidence does not always indicate correctness (documented in the appendix qualitative example).

Limitations

• Confidence reward is not a reliable proxy for correctness; optimization can converge to confidently incorrect answers.
• Per-task hyperparameter tuning uses grid search on validation splits; tuning for benchmarks without training sets relies on GSM8K proxy.
• Evaluated only on relatively small set of AIME problems (30 per year), so accuracy differences are small in absolute terms.

Experimental setup

Datasets: GSM8K ASDiv-Aug MATH-500 AIME2024 AIME2025 StrategyQA Date Understanding

Benchmarks: GSM8K MATH-500 ASDiv-Aug AIME2024 AIME2025 StrategyQA Date Understanding

Metrics: accuracy average inference time

Baselines: Zero-Shot CoT Zero-Shot CoT-Unk SoftCoT LatentSeek Genius SimpleRL-Zoo iCoT

Builds on: Coconut SoftCoT LatentSeek REINFORCE zero-shot chain-of-thought

Closest papers by meaning

• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.94)
• Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in its Latent Thoughts (ICLR 2026 · similarity 0.92)
• Optimizing Anytime Reasoning via Budget Relative Policy Optimization (NeurIPS 2025 · similarity 0.91)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.91)
• Hybrid Latent Reasoning via Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
• Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.91)