Research map / Large language models / Reasoning and chain of thought
Latent-Guided Reasoning: Empowering Small LLMs with Large-Model Thinking
Hanzhu Chen, Lin Yang, Jie Wang, Junhao Yan, Zhe Wang, Xize Liang, Jianye HAO
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes decoupling high-level cognitive planning from low-level linguistic realization so small LLMs can reason with large-model-style strategies (latent-guided reasoning).
Problem
LLM complex reasoning is computationally expensive because high-level solution planning and low-level step-by-step text generation are tightly coupled, limiting practical deployment.
Approach
Separates strategic planning from linguistic generation by using latent representations to guide small models' reasoning, aiming to transfer large-model thinking to small LLMs (details beyond the abstract not verified).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latent guidance from a 32B thinker to the small executor | LLaMA-3-8B · overall average of 5 benchmarks | accuracy: 70.3 -> 73.1 | Improved |
| add latent guidance | Qwen2-7B · OOD average | accuracy: 74.9 -> 78.5 | Improved |
| add latent guidance | Qwen2-0.5B · overall average | accuracy: 42.6 -> 44.3 | Improved |
| add latent guidance vs SFT | Qwen2.5-7B-Instruct · GSM8K | accuracy: 77.3 -> 80.5 | Improved |
| add latent guidance vs SFT | Qwen2.5-7B-Instruct · Odyssey-Math | accuracy: 15.5 -> 22.7 | Improved |
| add latent guidance vs SFT | Qwen2.5-7B-Instruct · average tokens on GSM8K | tokens: 235.4 -> 128.9 | Improved |
| add reconstruction loss to thinker training (ablation) | Latent Guidance · GSM8K | accuracy: 78.2 -> 80.5 | Improved |
| use 5 thought tokens instead of 3 (ablation) | Latent Guidance · GSM8K | accuracy: 79.4 -> 80.5 | Improved |
| increase thought tokens from 5 to 10 (ablation) | Latent Guidance · GSM8K | accuracy: 80.5 -> 81.1 | No clear effect |
| use 2-layer MLP projection instead of 1-layer linear (ablation) | Latent Guidance · GSM8K | accuracy: 79.0 -> 80.5 | Improved |
| use 3-layer MLP projection instead of 2-layer (ablation) | Latent Guidance · GSM8K | accuracy: 80.5 -> 80.4 | No clear effect |
| apply latent guidance to long-form QA | Qwen2.5-7B-Instruct · ELI5-Test (Biology) | GPT-4o avg score: 7.93 -> 8.45 | Improved |
Closest papers by meaning
- Thinking on the Fly: Test-Time Reasoning Enhancement via Latent Thought Policy Optimization (ICLR 2026 · similarity 0.92)
- Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.91)
- Training Language Models to Reason Efficiently (NeurIPS 2025 · similarity 0.91)
- Distilling LLM Agent into Small Models with Retrieval and Code Tools (NeurIPS 2025 · similarity 0.91)
- Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.91)
- THOUGHT PROPAGATION: AN ANALOGICAL APPROACH TO COMPLEX REASONING WITH LARGE LANGUAGE MODELS (ICLR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
