AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

Bridging Different Language Models and Generative Vision Models for Text-to-Image Generation

Shihao Zhao, Shaozhe Hao, Bojia Zi, Huaizhe Xu, Kwan-Yee K. Wong

ECCV 2024 · Image generation › Text-to-image diffusion · approach: Personalized and region-controlled generation · 14 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace CLIP text encoder with Llama-2LaVi-Bridge with SD V1.4 U-Net · T2I-CompBench Colorscore: 0.3578 -> 0.4859Improved
use larger T5 (Small to Base to Large)LaVi-Bridge with SD V1.4 U-Net · COCO2014 short promptsCLIP score: 0.3122 -> 0.3149 -> 0.3156Improved
replace CLIP with Llama-2 for long promptsLaVi-Bridge with SD V1.4 U-Net · Long promptsCLIP score: 0.3120 -> 0.3231Improved
replace SD U-Net with PixArt transformerLaVi-Bridge with T5-Large · COCO2014 short promptsaesthetic score: 5.881 -> 6.145Improved
replace LDM U-Net with SD V1.4 U-NetLaVi-Bridge with T5-Large · COCO2014 short promptsFID: 25.94 -> 23.11Improved
replace SD U-Net with PixArt transformerLaVi-Bridge with T5-Large · T2I-CompBench Colorscore: 0.3889 -> 0.3689Hurt
apply LaVi-Bridge training to the same CLIP and U-Net weights as SD V1.4Stable Diffusion V1.4 · COCO2014 short promptsFID: 20.32 -> 23.57Hurt
apply LaVi-Bridge training to the same CLIP and U-Net weights as SD V1.4Stable Diffusion V1.4 · T2I-CompBenchcompositional scores: Texture 0.4028 -> 0.4506Mixed
add LoRA to language and vision models (ablation)LaVi-Bridge T5-Large + SD U-Net · T2I-CompBench Colorscore: 0.2485 -> 0.3889Improved
use stacked feedforward adapter instead of a single linear layer (ablation)LaVi-Bridge T5-Large + SD U-Net · T2I-CompBench Shapescore: 0.3044 -> 0.3552Improved
add LoRA (ablation)LaVi-Bridge T5-Large + SD U-Net · COCO2014 short promptsFID: 22.35 -> 23.11Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy