AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

Do DALL-E and Flamingo Understand Each Other?

Hang Li, Jindong Gu, Rajat Koner, Sahand Sharifzadeh, Volker Tresp

ICCV 2023 · Image generation › Text-to-image diffusion · approach: Personalized and region-controlled generation · 13 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
choose the caption whose SD reconstruction is closest to the input imagefrozen BLIP + SD · NoCaps overallCIDEr: 74.1 -> 80.3Improved
filter SD images by BLIP text reconstructionSD · NoCapsFID: 32.37 -> 29.59Improved
finetune BLIP with SD reconstruction lossBLIP ViT-B · NoCaps overallCIDEr: 109.7 -> 111.8Improved
finetune BLIP-2 with SD reconstruction lossBLIP-2 · COCO Karpathy testCIDEr: 145.8 -> 146.4Improved
finetune BLIP ViT-L with SD reconstruction lossBLIP ViT-L · COCO Karpathy testSPICE: 24.3 -> 24.1No clear effect
finetune SD with BLIP text reconstruction lossSD · NoCapsCLIP image distance: 0.4039 -> 0.3978Improved
finetune SD with BLIP text reconstruction lossSD · NoCapsFID: 21.19 -> 21.96Hurt
add caption supervision loss L_TG alongside reconstruction loss (ablation)BLIP ViT-B · NoCaps overallCIDEr: 102.3 -> 111.8Improved
add image reconstruction loss L_IR (ablation)BLIP ViT-B · NoCaps overallCIDEr: 109.7 -> 111.8Improved
add text reconstruction loss L_TR (ablation)SD · NoCapsFID: 24.32 -> 21.96Improved
finetune SD with MSE loss on COCO pairs onlySD · NoCapsFID: 21.19 -> 24.32Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy