Research map / Image generation / Text-to-image diffusion
BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing
Dongxu Li, Junnan Li, Steven Hoi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BLIP-2 multimodal representation pre-training (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | DINO: 0.521 -> 0.566 | Improved |
| add BLIP-2 multimodal representation pre-training (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | CLIP-T: 0.290 -> 0.299 | Improved |
| train (unfreeze) the diffusion text encoder (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | DINO: 0.568 -> 0.566 | Mixed |
| train (unfreeze) the diffusion text encoder (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | CLIP-T: 0.288 -> 0.299 | Improved |
| add subject category text to the multimodal encoder (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | DINO: 0.565 -> 0.566 | No clear effect |
| add random subject prompt dropping (15%) in pre-training (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | DINO: 0.559 -> 0.566 | Improved |
| add random subject prompt dropping (15%) in pre-training (ablation) | BLIP-Diffusion · DreamBench zero-shot, 250K steps | CLIP-T: 0.291 -> 0.299 | Improved |
| fine-tune for under 80 steps vs zero-shot (ablation) | BLIP-Diffusion · DreamBench | DINO: 0.594 -> 0.670 | Improved |
| fine-tune for under 80 steps vs zero-shot (ablation) | BLIP-Diffusion · DreamBench | CLIP-T: 0.300 -> 0.302 | No clear effect |
| pre-trained subject representation vs DreamBooth with 100 steps | BLIP-Diffusion · DreamBench | DINO: 0.396 -> 0.670 | Improved |
| BLIP-Diffusion fine-tuned vs full DreamBooth | BLIP-Diffusion · DreamBench | DINO: 0.668 -> 0.670 | No clear effect |
| increase subject representation pre-training steps (ablation) | BLIP-Diffusion · DreamBench | CLIP-I | Improved |
Closest papers by meaning
- DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation (ICLR 2025 · similarity 0.88)
- Mastering Text-to-Image Diffusion: Recaptioning, Planning, and Generating with Multimodal LLMs (ICML 2024 · similarity 0.88)
- Compositional Text-to-Image Generation with Dense Blob Representations (ICML 2024 · similarity 0.87)
- R&B: Region and Boundary Aware Zero-shot Grounded Text-to-image Generation (ICLR 2024 · similarity 0.87)
- Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator (CVPR 2025 · similarity 0.87)
- AP-Adapter: Improving Generalization of Automatic Prompts on Unseen Text-to-Image Diffusion Models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
