Research map / Large language models / Reasoning and chain of thought

LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Search and self-correction in reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LaDiR, which encodes reasoning steps into VAE latent thought-token blocks and runs a flow-matching latent diffusion model over them (with blockwise causal attention, rollout training, and diversity guidance) to perform text reasoning on top of an existing LLM.

Problem

Autoregressive chain-of-thought decoding struggles to revise earlier reasoning holistically and yields low solution diversity; existing latent and diffusion language models do not target multi-step reasoning accuracy.

Approach

A beta-VAE builds a sentence-level latent space of thought tokens; a flow-matching latent diffusion reasoner denoises these blocks with intra-block bidirectional and inter-block causal attention, followed by answer decoding. Diversity is promoted by a repulsion force across a batch during denoising, and rollout training backpropagates answer loss through the denoising trajectory.

Key results

• On math benchmarks with LLaMA 3.1 8B, LaDiR reaches 43.5 average Pass@1 versus 42.6 for the best SFT variant and 42.0 for TaH+ (average over 7 benchmarks, per Table 1).
• On Countdown CD-4, LaDiR achieves 76.6 Pass@1 and 96.4 Pass@100 versus 46.7 and 65.3 for LLaMA 8B SFT, with diversity 7.3 vs 3.0.
• On code generation with Qwen3-8B-Base, LaDiR averages 74.5 versus 69.3 for AR SFT and 71.8 for TaH+, with HumanEval+ at 84.2.

Surprising findings

• Coconut fails to surpass AR CoT SFT on math benchmarks in the reported results.
• Increasing the diffusion denoising steps and latent block size beyond a point degrades downstream accuracy, indicating redundancy makes latent prediction harder.

Limitations

• Math results use a flow-matching model with multiple stages and reports Pass@100 with exact string matching, which may not reflect correctness in all settings.
• Latent block size and diversity hyperparameters require tuning; excessive settings hurt accuracy.

Experimental setup

Datasets: DART-MATH LingCode Countdown synthetic dataset

Benchmarks: MATH GSM8K Fresh-Gaokao-Math-2023 DeepMind-Math College-Math OlympiadBench-Math TheoremQA MBPP MBPP+ HumanEval HumanEval+ Countdown

Metrics: Pass@1 Pass@100 diversity average accuracy

Baselines: CoT SFT Coconut CODI Soft Thinking TaH+ LLaDA 8B PLANNER LD4LG MGDM Ouro 2.6B

Builds on: Coconut block diffusion flow matching beta-VAE FlowGRPO

Closest papers by meaning

• Token Assorted: Mixing Latent and Text Tokens for Improved Language Model Reasoning (ICML 2025 · similarity 0.91)
• Why Struggle with Continuous Latents? Interpretable Discrete Latent Reasoning via Rendered Compression (NeurIPS 2026 · similarity 0.91)
• Reasoning via Test-Time Instance-Level Policy Gradient in Latent Space (NeurIPS 2026 · similarity 0.90)
• Efficient Post-Training Refinement of Latent Reasoning in Large Language Models (AAAI 2026 · similarity 0.90)
• Think in Latent, Explain in Language: Self-Explainable Latent Reasoning (ICML 2026 · similarity 0.90)
• Geometric Latent Reasoning Induces Shorter Generations in LLMs (NeurIPS 2026 · similarity 0.90)