AI Paper Atlas

Research map / Image generation / Text-to-image diffusion

The CLIP Model is Secretly an Image-to-Prompt Converter

Yuxuan Ding, Chunna Tian, Haoxuan Ding, Lingqiao Liu

NeurIPS 2023 · Image generation › Text-to-image diffusion · approach: Text-driven synthesis and semantic guidance · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
map CLIP image embedding to T-space with inverse projection matrix instead of C-spaceCLIP ViT-L/14 · ImageNet valAcc@1: 71.41 -> 69.48No clear effect
same inverse matrix transferCLIP · image retrievalIR@1: 55.54 -> 54.82No clear effect
training-free SD-IPC vs text-based SD generationStable Diffusion · MSCOCO valFID: 23.65 -> 24.78No clear effect
SD-IPC vs text generation on CLIP-ScoreStable Diffusion · MSCOCO valCLIP-Score: 70.15 -> 73.57Improved
fine-tune both CLIP prompts and U-Net instead of SD-IPC alone (ablation)SD-IPC · image variationDINO: 44.60 -> 52.03Improved
train only CLIP prompts (C) (ablation)SD-IPC-FT · text-edited variationDINO: 31.09 -> 29.10Hurt
train only U-Net cross-attention (U) (ablation)SD-IPC-FT · text-edited variationDINO: 31.09 -> 35.21Improved
fine-tune both components rather than one (ablation)SD-IPC-FT · text-edited variationDINO: 35.21 -> 40.28Improved
fine-tune both rather than U only (ablation)SD-IPC-FT · image variationCLIP-T: 26.17 -> 25.90Hurt
fine-tuning for editingSD-IPC · DreamBooth editingCLIP-T: 26.84 -> 28.69Improved
SD-IPC vs SD-R editingSD-IPC · DreamBooth editingCLIP-T: 26.01 -> 26.84Improved
SD-IPC-CT vs DreamBooth on editingSD-IPC-CT · DreamBooth benchmarkCLIP-T: 25.81 -> 28.14Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy