Research map / Image generation / Text-to-image diffusion
Unleashing Text-to-Image Diffusion Models for Visual Perception
Wenliang Zhao, Yongming Rao, Zuyan Liu, Benlin Liu, Jie Zhou, Jiwen Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add class-name text prompt (ablation) | VPD · ADE20K (8K iters) | mIoU: 46.9 -> 47.1 | Improved |
| add class-name text prompt (ablation) | VPD · ADE20K (4K iters) | mIoU: 41.5 -> 42.0 | Improved |
| add text adapter after CLIP text encoder (ablation) | VPD · ADE20K (4K iters) | mIoU: 42.0 -> 42.9 | Improved |
| add text adapter after CLIP text encoder (ablation) | VPD · ADE20K (8K iters) | mIoU: 47.1 -> 48.0 | Improved |
| add cross-attention maps as semantic guidance (up+down) (ablation) | VPD · ADE20K (8K iters) | mIoU: 48.0 -> 48.7 | Improved |
| use cross-attention maps from middle blocks (ablation) | VPD · ADE20K (8K iters) | mIoU: 48.0 -> 47.8 | Hurt |
| use cross-attention maps from up blocks rather than down blocks (ablation) | VPD · ADE20K (8K iters) | mIoU: 48.2 -> 48.5 | Improved |
| combine up and down cross-attention maps (ablation) | VPD · ADE20K (8K iters) | mIoU: 48.5 -> 48.7 | Improved |
| use cross-attention maps of larger size (A64 vs A32) at 4K iterations (ablation) | VPD · ADE20K (4K iters) | mIoU single-scale: 43.1 -> 43.9 | Improved |
| use A32 instead of A64 maps at 8K iterations (ablation) | VPD · ADE20K (8K iters) | mIoU single-scale: 47.7 -> 48.7 | Improved |
| more pre-trained iterations of Stable-Diffusion (SD-1-1 to SD-1-5) (ablation) | VPD · ADE20K (8K iters) | mIoU: improved by more than 4 | Improved |
| swap pre-trained backbone for VPD under 1-epoch fast schedule (vs LAVT) | LAVT · RefCOCO val | oIoU: 52.56 -> 63.04 | Improved |
Closest papers by meaning
- DEEM: Diffusion models serve as the eyes of large language models for image perception (ICLR 2025 · similarity 0.93)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.92)
- What Matters When Repurposing Diffusion Models for General Dense Perception Tasks? (ICLR 2025 · similarity 0.92)
- Text-Image Alignment for Diffusion-Based Perception (CVPR 2024 · similarity 0.92)
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.91)
- Scaling Properties of Diffusion Models For Perceptual Tasks (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
