Research map / Large language models / Reasoning and chain of thought

Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression

NeurIPS 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Discrete Latent Reasoning (DLR), which renders CoT traces as images, quantizes DeepSeek-OCR2 visual features into a discrete latent vocabulary, and trains LLMs/LMMs over text-plus-latent tokens via alignment, SFT, and GRPO with a process reward.

Problem

Continuous latent reasoning is hard to train and is unstable and uninterpretable because continuous latent states lack discrete anchors for supervision, credit assignment and inspection.

Approach

Convert continuous latent reasoning states into discrete tokens via a vector-quantized codebook learned from rendered CoT images (with an OCR decoder for interpretability), expand the LLM vocabulary and output head with these tokens, and train with standard next-token prediction plus RL using a decoded-trace process reward.

Key results

• On Qwen3-VL-4B, DLR (Full) averages 63.2% accuracy over four grade-school benchmarks with ~5.4 latent tokens, versus 45.1% for CoLaR and 55.4% for RoT.
• On MATH-500 with Qwen3-VL-8B, DLR (Full) reaches 54.0% accuracy vs 37.8% for RoT, using ~29.5 latent tokens vs 64 for RoT.
• On LLaMA-3.2-1B, DLR achieves 51.3% average accuracy vs 45.6% for ReGuLaR.

Surprising findings

• Skipping the latent SFT stage causes performance to collapse (e.g., GSM8K-Aug accuracy drops to 21.4%), indicating RL alone cannot learn latent distributions from sparse outcome rewards.

Limitations

• Decoded latent traces occasionally hallucinate details such as names or units.
• Evaluation focuses on arithmetic and competition math benchmarks; generality to other reasoning domains is not shown in the provided text.

Experimental setup

Datasets: GSM8K-Aug MATH MathX-5M GSM8K GSM-Hard SVAMP MultiArith MATH-500

Benchmarks: GSM8K GSM-Hard SVAMP MultiArith MATH-500

Metrics: Pass@1 accuracy average reasoning token length (#L)

Baselines: iCoT Coconut CODI CoLaR ReGuLaR RoT CoT-SFT

Builds on: Coconut CODI CoLaR ReGuLaR DeepSeek-OCR GRPO

Closest papers by meaning

• Geometric Latent Reasoning Induces Shorter Generations in LLMs (NeurIPS 2026 · similarity 0.92)
• KaVa: Latent Reasoning via Compressed KV-Cache Distillation (ICLR 2026 · similarity 0.91)
• Think Silently, Think Fast: Dynamic Latent Compression of LLM Reasoning Chains (NeurIPS 2025 · similarity 0.91)
• Think in Latent, Explain in Language: Self-Explainable Latent Reasoning (ICML 2026 · similarity 0.91)
• Leveraging Latent Visual Reasoning in Silence (NeurIPS 2026 · similarity 0.91)
• LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning (ICLR 2026 · similarity 0.91)