Research map / Large language models / Reasoning and chain of thought

Think in Latent, Explain in Language: Self-Explainable Latent Reasoning

ICML 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SELR, which trains one latent-reasoning model with an answer loss plus a CoT-decoding loss so the same model can decode its own latent thoughts into text, evaluated on LLaMA-3.2-1B (LLM) and Qwen2.5-VL-3B (VLM).

Problem

Latent reasoning compresses chain-of-thought into continuous embeddings, which makes the reasoning opaque and hard to supervise; existing fixes either leave latents unreadable (Coconut) or need separate post-hoc decoders (Heima).

Approach

A single model is trained jointly with an Answer Loss on latent-conditioned answers and a CoT Loss that makes the same model decode its own latent thoughts into text (full-CoT or per-step), so explanation and reasoning share parameters without an external decoder.

Key results

• On Qwen2.5-VL-3B, SELR (Multi, Exponential) averages 63.31% accuracy vs 62.86% base while cutting output tokens from 49.92 to 13.47 (-73%).
• On LLaMA-3.2-1B GSM8k, SELR (Multi) reaches 42.46% vs Coconut 30.83% and CoLaR 40.1%, with 6 latent tokens.
• Heima decoding on MathVista consistency scores 0.1671 vs SELR (Single) 0.2097; Heima's VLM average drops to 60.94%.

Surprising findings

• Training with only the Answer Loss degrades VLM accuracy below the base model, while adding any CoT loss reverses this.
• CoT-SFT constrained to a 6-token budget collapses to 10.31% on GSM8k, versus 42.46% for SELR, suggesting latent tokens encode reasoning more densely.

Limitations

• Faithfulness of decoded CoT is not guaranteed; decoded reasoning can be correct while the final answer is wrong.
• Fixed latent length is required because a learned <eot> stopping mechanism was hard to train.
• MMVet scores drop for multi-stage variants, attributed to loss of long-form and multi-step generation capability.

Experimental setup

Datasets: LLaVA-CoT-100k GSM8k-Aug GSM8k VisualWebInstruct-Verified

Benchmarks: MMStar MMBench MMVet MathVista AI2D HallusionBench GSM8k SVAMP GSM-Hard MultiArith

Metrics: accuracy token count BLEU-4 METEOR ROUGE-L BERTScore GPT-4o similarity score consistency score

Baselines: Coconut CoLaR CoT-SFT Heima Qwen2.5-VL-3B-Instruct Qwen2.5-VL-3B-Instruct-SFT

Builds on: Coconut Heima CoLaR LLaVA-CoT

Closest papers by meaning

• Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression (NeurIPS 2026 · similarity 0.91)
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning (ICLR 2026 · similarity 0.90)
• SLVR: Structured Latent Visual Reasoning via Human-like Reasoning Flows (NeurIPS 2026 · similarity 0.89)
• Latent Thoughts Tuning: Bridging Context and Reasoning with Fused Information in Latent Tokens (ICML 2026 · similarity 0.89)
• KaVa: Latent Reasoning via Compressed KV-Cache Distillation (ICLR 2026 · similarity 0.89)
• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.89)