Research map / Image generation / Autoregressive and latent generation
Coevolutionary Continuous Discrete Diffusion: Make Your Diffusion Language Model a Latent Reasoner
Cai Zhou, Chenxiao Yang, Yi Hu, Chenyu Wang, Chubin Zhang, Muhan Zhang, Lester Mackey, Tommi Jaakkola, Stephen Bates, Dinghuai Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use contextualized Qwen3-Embedding latents with joint continuous-discrete diffusion instead of discrete-only masked diffusion | CCDD-MDiT (same params and FLOPs) · LM1B validation | validation perplexity: 39.17 -> 29.22 | Improved |
| scale architecture from MDiT to MMDiT (ablation) | CCDD with Qwen3-Embedding · LM1B validation | validation perplexity: 29.22 -> 25.76 | Improved |
| use MoEDiT architecture instead of MDiT (ablation) | CCDD with Qwen3-Embedding · OWT validation (Qwen-2 tokenizer) | validation perplexity: 29.18 -> 21.90 | Improved |
| use MMDiT instead of MoEDiT (ablation) | CCDD with Qwen3-Embedding · OWT validation (Qwen-2 tokenizer) | validation perplexity: 21.90 -> 26.59 | Hurt |
| use even simple RoBERTa embeddings as continuous space | CCDD-MoEDiT · OWT validation (GPT-2 tokenizer) | validation perplexity: 27.39 -> 24.56 | Improved |
| add representation classifier-free guidance, w=1.0 (ablation) | CCDD-MoEDiT · OWT generation | generative NLL: 9.06 -> 8.38 | Improved |
| increase guidance scale from 1.0 to 1.5 (ablation) | CCDD-MoEDiT · OWT generation | generative NLL: 8.38 -> 8.25 | Improved |
| use CCDD with only 8 denoising steps versus MDLM with 256 steps | CCDD · OWT few-step generation | generative NLL | Improved |
| use CCDD training to reach MDLM final perplexity | CCDD-MDiT · LM1B validation | training iterations to 39.17 PPL: 1000k -> 40k iterations | Improved |
| use contextualized last-layer latents instead of first-layer token-wise latents (ablation) | continuous diffusion on Qwen3-Embedding-0.6B · LM1B (500k steps) | representation MSE vs token cross-entropy | Mixed |
| add looped depth to LT (T=2 to T=3) (ablation) | looped transformer 6M · Countdown | accuracy: 60.6 -> 68.2 | Improved |
| add looped depth to CCDD (T=2 to T=3) (ablation) | CCDD 6M · Countdown | accuracy: 67.8 -> 73.7 | Improved |
Closest papers by meaning
- Non-Markovian Discrete Diffusion with Causal Language Models (NeurIPS 2025 · similarity 0.91)
- On the Reasoning Abilities of Masked Diffusion Language Models (ICLR 2026 · similarity 0.91)
- From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs (NeurIPS 2026 · similarity 0.91)
- Continuous Latent Diffusion Language Model (NeurIPS 2026 · similarity 0.90)
- LangFlow: Continuous Diffusion Rivals Discrete in Language Modeling (NeurIPS 2026 · similarity 0.90)
- Encoder-Decoder Diffusion Language Models for Efficient Training and Inference (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
