AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

BLIP-Diffusion: Pre-trained Subject Representation for Controllable Text-to-Image Generation and Editing

Dongxu Li, Junnan Li, Steven Hoi

NeurIPS 2023 · Image generation › Text-to-image diffusion · approach: Compositional text-to-image guidance · 65 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add BLIP-2 multimodal representation pre-training (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsDINO: 0.521 -> 0.566Improved
add BLIP-2 multimodal representation pre-training (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsCLIP-T: 0.290 -> 0.299Improved
train (unfreeze) the diffusion text encoder (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsDINO: 0.568 -> 0.566Mixed
train (unfreeze) the diffusion text encoder (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsCLIP-T: 0.288 -> 0.299Improved
add subject category text to the multimodal encoder (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsDINO: 0.565 -> 0.566No clear effect
add random subject prompt dropping (15%) in pre-training (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsDINO: 0.559 -> 0.566Improved
add random subject prompt dropping (15%) in pre-training (ablation)BLIP-Diffusion · DreamBench zero-shot, 250K stepsCLIP-T: 0.291 -> 0.299Improved
fine-tune for under 80 steps vs zero-shot (ablation)BLIP-Diffusion · DreamBenchDINO: 0.594 -> 0.670Improved
fine-tune for under 80 steps vs zero-shot (ablation)BLIP-Diffusion · DreamBenchCLIP-T: 0.300 -> 0.302No clear effect
pre-trained subject representation vs DreamBooth with 100 stepsBLIP-Diffusion · DreamBenchDINO: 0.396 -> 0.670Improved
BLIP-Diffusion fine-tuned vs full DreamBoothBLIP-Diffusion · DreamBenchDINO: 0.668 -> 0.670No clear effect
increase subject representation pre-training steps (ablation)BLIP-Diffusion · DreamBenchCLIP-IImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy