Research map / Image generation / Fast diffusion sampling
Logit-Conditioned Diffusion Decoding for Frozen Discrete-Token VLMs
Ji Woo Hong, Hee Suk Yoon, Gwanhyeong Koo, Eunseop Yoon, SooHwan Eom, Qi Dai, Chong Luo, Chang Yoo
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a logit-conditioned diffusion decoder that improves the visual fidelity of frozen discrete-token VLMs by replacing the VQ-VAE decoding stage, without retraining the foundation model.
Problem
Discrete-token VLMs that generate images via codebook indices are limited in visual fidelity by the VQ-VAE decoder; prior fixes require replacing the tokenizer or retraining foundation-scale models.
Approach
Conditions a diffusion decoder on the frozen VLM's output logits over codebook indices, so the bottleneck VQ-VAE decoding stage is upgraded while the discrete-token VLM stays frozen.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace VQ-VAE decoder with DCDD logit-conditioned diffusion decoder | Janus-Pro 7B · MJHQ-30K | gFID: 14.29 -> 9.42 | Improved |
| replace VQ-VAE decoder with DCDD | Janus-Pro 7B · ImageNet-1K val reconstruction | rFID: 2.8511 -> 0.7535 | Improved |
| replace VQ-VAE decoder with DCDD | Janus-Pro 7B · GenEval/DPG-Bench/WISE | semantic alignment: GenEval 77.24 -> 78.31; DPG 86.94 -> 87.64 | No clear effect |
| condition diffusion decoder on full logit distribution instead of argmax codes (ablation) | DCDD · MJHQ-30K | gFID: 11.0837 -> 9.4160 | Improved |
| use diffusion decoder with argmax codes instead of VQ-VAE decoder (ablation) | Janus-Pro 7B · MJHQ-30K | gFID: 14.29 -> 11.08 | Improved |
| retain only top-1 token of logit distribution (ablation) | DCDD · MJHQ-30K Animals | gFID: 31.6866 -> 33.2015 | Hurt |
| include second-ranked token in conditioning (ablation) | DCDD · MJHQ-30K Animals | gFID: 33.2015 -> 27.0649 | Improved |
| add tokens ranked 3-8 and tail (ablation) | DCDD · MJHQ-30K Animals | gFID: 27.0649 -> 25.5681 -> 25.1060 | Improved |
| use FLUX.1 Kontext image-to-image refiner instead of logit conditioning (ablation) | VQ-VAE output · MJHQ-30K | gFID: 9.4160 (DCDD) vs 10.6678 (I2I generic) | Hurt |
| apply 7B-trained DCDD to Janus-Pro 1B without retraining | Janus-Pro 1B · MJHQ-30K Animals | gFID: 35.14 -> 26.09 | Improved |
| apply DCDD to LlamaGen-XL (Otsu rank 1 logits) | LlamaGen-XL · MJHQ-30K Animals | gFID: 74.20 -> 74.66 | No clear effect |
| use DCDD decoder with 30 denoising steps | Janus-Pro 7B · MJHQ-30K end-to-end | latency (s): 11.86 -> 15.56 | Hurt |
Closest papers by meaning
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.89)
- ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.89)
- Bifrost-1: Bridging Multimodal LLMs and Diffusion Models with Patch-level CLIP Latents (NeurIPS 2025 · similarity 0.89)
- Faster Diffusion: Rethinking the Role of the Encoder for Diffusion Model Inference (NeurIPS 2024 · similarity 0.89)
- Epsilon-VAE: Denoising as Visual Decoding (ICML 2025 · similarity 0.89)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
