Research map / Image generation / Fast diffusion sampling
Context Tokens are Anchors: Understanding the Repetition Curse in Diffusion MLLMs from an Information Flow Perspective
Qiyan Zhao, Xiaofeng Zhang, Shuochen Chang, Qianyu Chen, Xiaosong Yuan, Xuhang Chen, Luoqi Liu, Jiajun Zhang, Xu-Yao Zhang, Da-Han Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| enable dllm-cache on LLaDA-V | LLaDA-V · COCO caption repetition (512) | SRR: 6.9 -> 82.3 | Hurt |
| add CTEV to cached model (ablation) | LLaDA-V + dllm-cache · COCO caption (512) | ARR: 14.3 -> 3.2 | Improved |
| add CTAE to cached model (ablation) | LLaDA-V + dllm-cache · COCO caption (512) | ARR: 14.3 -> 2.9 | Improved |
| combine CTEV and CTAE (ablation) | LLaDA-V + dllm-cache · COCO caption (512) | ARR: 3.2/2.9 -> 1.2 | Improved |
| add CoTA to dllm-cache | LLaDA-V · MathVista | score: 54.9 -> 59.7 | Improved |
| add CoTA to dllm-cache | LLaDA-V · LLaVA-W | score: 63.2 -> 69.9 | Improved |
| add CoTA to dllm-cache | LLaDA-V · DocVQA | accuracy: 82.1 -> 84.1 | Improved |
| add CoTA to dllm-cache | LLaDA-V · MMBench | accuracy: 83.0 -> 83.1 | No clear effect |
| CoTA vs uncached LLaDA-V | LLaDA-V · MMStar / SEED-Image | accuracy: 60.1 -> 59.3; 74.8 -> 73.9 | Hurt |
| increase context tokens from 3 to 5 (ablation) | CoTA · MathVerse | ARR: 1.2 -> 3.3 | Hurt |
| restrict entropy to layers 26-30 vs earlier layers (ablation) | CTEV · COCO caption (512) | SRR: 81% (layers 1-10) -> 10% (layers 26-30) | Improved |
| use Gaussian decay instead of ALiBi bias in CTAE (ablation) | CTAE · COCO caption | ARR: 3.9% -> 1.8% | Improved |
Closest papers by meaning
- Interpreting the Repeated Token Phenomenon in Large Language Models (ICML 2025 · similarity 0.88)
- Understanding In-Context Learning from Repetitions (ICLR 2024 · similarity 0.88)
- EntropyCache: Decoded Token Entropy Guided KV Caching for Diffusion Language Models (NeurIPS 2026 · similarity 0.87)
- From Next-Token to Next-Block: A Principled Adaptation Path for Diffusion LLMs (NeurIPS 2026 · similarity 0.87)
- L$^2$M: Mutual Information Scaling Law for Long-Context Language Modeling (NeurIPS 2025 · similarity 0.87)
- Polestar: Drift-Aware Cache Calibration and Token Commitment for Efficient Inference of Diffusion LLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
