AI Paper Atlas

Research map / Large language models / Reasoning and chain of thought

Latent-Guided Reasoning: Empowering Small LLMs with Large-Model Thinking

Hanzhu Chen, Lin Yang, Jie Wang, Junhao Yan, Zhe Wang, Xize Liang, Jianye HAO

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes decoupling high-level cognitive planning from low-level linguistic realization so small LLMs can reason with large-model-style strategies (latent-guided reasoning).

Problem

LLM complex reasoning is computationally expensive because high-level solution planning and low-level step-by-step text generation are tightly coupled, limiting practical deployment.

Approach

Separates strategic planning from linguistic generation by using latent representations to guide small models' reasoning, aiming to transfer large-model thinking to small LLMs (details beyond the abstract not verified).

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add latent guidance from a 32B thinker to the small executorLLaMA-3-8B · overall average of 5 benchmarksaccuracy: 70.3 -> 73.1Improved
add latent guidanceQwen2-7B · OOD averageaccuracy: 74.9 -> 78.5Improved
add latent guidanceQwen2-0.5B · overall averageaccuracy: 42.6 -> 44.3Improved
add latent guidance vs SFTQwen2.5-7B-Instruct · GSM8Kaccuracy: 77.3 -> 80.5Improved
add latent guidance vs SFTQwen2.5-7B-Instruct · Odyssey-Mathaccuracy: 15.5 -> 22.7Improved
add latent guidance vs SFTQwen2.5-7B-Instruct · average tokens on GSM8Ktokens: 235.4 -> 128.9Improved
add reconstruction loss to thinker training (ablation)Latent Guidance · GSM8Kaccuracy: 78.2 -> 80.5Improved
use 5 thought tokens instead of 3 (ablation)Latent Guidance · GSM8Kaccuracy: 79.4 -> 80.5Improved
increase thought tokens from 5 to 10 (ablation)Latent Guidance · GSM8Kaccuracy: 80.5 -> 81.1No clear effect
use 2-layer MLP projection instead of 1-layer linear (ablation)Latent Guidance · GSM8Kaccuracy: 79.0 -> 80.5Improved
use 3-layer MLP projection instead of 2-layer (ablation)Latent Guidance · GSM8Kaccuracy: 80.5 -> 80.4No clear effect
apply latent guidance to long-form QAQwen2.5-7B-Instruct · ELI5-Test (Biology)GPT-4o avg score: 7.93 -> 8.45Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy