Research map / Large language models / Reasoning and chain of thought

Latent Thinking Optimization: Your Latent Reasoning Language Model Secretly Encodes Reward Signals in its Latent Thoughts

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that latent thoughts of Huginn-3.5B separate correct from incorrect answers, and proposes Latent Thinking Optimization (LTO), which uses a latent classifier as a Latent Reward Model to resample latent trajectories via acceptance-rejection sampling.

Problem

Latent reasoning models such as Huginn-3.5B think in unsupervised hidden-state trajectories that are hard to interpret or supervise, so it is unclear whether their latent thoughts are correct and how to improve them.

Approach

Train a lightweight Transformer probe over the sequence of latent thoughts to predict answer correctness (a Latent Reward Model), then reweight/resample latent trajectories from the base policy via a KL-regularized, acceptance-rejection scheme so that higher-reward trajectories are preferentially kept, with no parameter updates to the base model.

Key results

• On Huginn-3.5B, LTO with LRM reaches 0.385 on GSM8K vs 0.326 base model and 0.342 best baseline (self-correction with confidence).
• The latent classifier reaches ROC-AUC close to 1.0 on SVAMP and around 0.8 on MBPP.
• LTO with N=20 improves general LLMs by up to 103% over base model (e.g., Llama-2-7B GSM8K 0.223 to 0.389).

Surprising findings

• Standard majority voting yields little to no benefit on Huginn-3.5B, while LRM-weighted majority voting gives substantial gains.
• Verbal self-correction with verbal evaluation often hurts performance (e.g., GSM8K 0.262 vs 0.326 base).

Limitations

• Latent classifier is imperfect on harder tasks such as MBPP (ROC-AUC around 0.8), so LTO can exploit reward-model errors without KL regularization.
• Cross-domain generalization is not yet full; the general LRM only matches domain-specific LRMs in some settings.
• Correctness is defined by final-answer agreement, which is a coarse proxy for latent-thought quality.

Experimental setup

Datasets: GSM8K SVAMP GSM-Symbolic CommonsenseQA MBPP

Benchmarks: GSM8K GSM-Symbolic SVAMP CommonsenseQA MBPP

Metrics: answer correctness rate ROC-AUC accuracy

Baselines: Majority Voting Self-Correction w. Confidence Score Self-Correction w. Verbal Evaluation Latent Thinking Correction w. CoE-R Latent Thinking Correction w. CoE-C Weighted Majority Voting w. LRM

Builds on: Huginn-3.5B KL-regularized policy optimization process reward models probing

Closest papers by meaning

• Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.92)
• Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens (ICML 2026 · similarity 0.90)
• Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.89)
• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.89)
• Rethinking Thinking Tokens: LLMs as Improvement Operators (ICML 2026 · similarity 0.89)
• Don’t Think Longer, Think Wisely: Optimizing Thinking Dynamics for Large Reasoning Models (NeurIPS 2025 · similarity 0.89)