AI Paper Atlas

Research map / Image generation / Fast diffusion sampling

RLEG: Vision-Language Representation Learning with Diffusion-based Embedding Generation

Liming Zhao, Kecheng Zheng, Yun Zheng, Deli Zhao, Jingren Zhou

ICML 2023 · Image generation › Fast diffusion sampling · approach: Diffusion models as perception tools

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add diffusion-based embedding generator guidance (vs CLIP baseline)RLEG on YFCC15M · ImageNet-1K zero-shotTop-1 Acc.: +9.0%Improved
add multiple (K=4) generated samples vs single sample (K=1) (ablation)RLEG generator guidance · ImageNet-1K zero-shotTop-1 Acc.: 36.7% to 39.1%Improved
add generator guidanceRLEG on COCO and Flickr30K retrieval · COCO and Flickr30K zero-shot retrievalR@1: +3.5%, +6.2%, +8.4%, +10.7%Improved
add generator guidanceRLEG backbone for DALL-E 2-style image generation · ImageNet and COCO 64x64 generationFID: 11.5 to 9.3 (ImageNet); 15.7 to 13.1 (COCO)Improved
add generator guidanceRLEG on COCO object detection and segmentation · zero-shot COCO detection AP and segmentation mIoUAP / mIoU: +4.2 AP; +3.4 mIoUImproved
increase number of samples K from 0 to 1, 2, 4 (ablation)RLEG on YFCC3M/CC3M/CC12M/YFCC15M · ImageNet-1K zero-shot top-1Top-1 Acc.: consistent gains across dataset sizes, e.g. YFCC3M 16.4 to 24.3 at K=4Improved
reduce DDIM sampling steps from 10 to 5 or raise to 50 (ablation)RLEG generator sampling · ImageNet-1K zero-shot top-1Top-1 Acc.: 38.7 (5 steps), 39.1 (10), 39.2 (50)No clear effect
replace two-layer MLP projector with identity mapping (ablation)RLEG projector · ImageNet-1K zero-shot top-1Top-1 Acc.: drops from 39.1% to 38.4%Hurt
vary classifier-free guidance weight w (0.1, 1.0, 2.0, 5.0) (ablation)RLEG generator sampling · ImageNet-1K zero-shot top-1Top-1 Acc.: 32.4, 38.3, 39.1, 38.8Mixed
add generator guidance with LAION-400M and ViT-L/14 encoderRLEG · ImageNet-1K zero-shot top-1Top-1 Acc.: +4.5 (75.3 to 79.8)Improved
train RLEG for 16 epochs vs CLIP for 32 epochs (matched wall-clock)RLEG · ImageNet-1K zero-shot top-1Top-1 Acc.: 33.2% vs 30.1%Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy