Research map / Image generation / Fast diffusion sampling
RLEG: Vision-Language Representation Learning with Diffusion-based Embedding Generation
Liming Zhao, Kecheng Zheng, Yun Zheng, Deli Zhao, Jingren Zhou
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add diffusion-based embedding generator guidance (vs CLIP baseline) | RLEG on YFCC15M · ImageNet-1K zero-shot | Top-1 Acc.: +9.0% | Improved |
| add multiple (K=4) generated samples vs single sample (K=1) (ablation) | RLEG generator guidance · ImageNet-1K zero-shot | Top-1 Acc.: 36.7% to 39.1% | Improved |
| add generator guidance | RLEG on COCO and Flickr30K retrieval · COCO and Flickr30K zero-shot retrieval | R@1: +3.5%, +6.2%, +8.4%, +10.7% | Improved |
| add generator guidance | RLEG backbone for DALL-E 2-style image generation · ImageNet and COCO 64x64 generation | FID: 11.5 to 9.3 (ImageNet); 15.7 to 13.1 (COCO) | Improved |
| add generator guidance | RLEG on COCO object detection and segmentation · zero-shot COCO detection AP and segmentation mIoU | AP / mIoU: +4.2 AP; +3.4 mIoU | Improved |
| increase number of samples K from 0 to 1, 2, 4 (ablation) | RLEG on YFCC3M/CC3M/CC12M/YFCC15M · ImageNet-1K zero-shot top-1 | Top-1 Acc.: consistent gains across dataset sizes, e.g. YFCC3M 16.4 to 24.3 at K=4 | Improved |
| reduce DDIM sampling steps from 10 to 5 or raise to 50 (ablation) | RLEG generator sampling · ImageNet-1K zero-shot top-1 | Top-1 Acc.: 38.7 (5 steps), 39.1 (10), 39.2 (50) | No clear effect |
| replace two-layer MLP projector with identity mapping (ablation) | RLEG projector · ImageNet-1K zero-shot top-1 | Top-1 Acc.: drops from 39.1% to 38.4% | Hurt |
| vary classifier-free guidance weight w (0.1, 1.0, 2.0, 5.0) (ablation) | RLEG generator sampling · ImageNet-1K zero-shot top-1 | Top-1 Acc.: 32.4, 38.3, 39.1, 38.8 | Mixed |
| add generator guidance with LAION-400M and ViT-L/14 encoder | RLEG · ImageNet-1K zero-shot top-1 | Top-1 Acc.: +4.5 (75.3 to 79.8) | Improved |
| train RLEG for 16 epochs vs CLIP for 32 epochs (matched wall-clock) | RLEG · ImageNet-1K zero-shot top-1 | Top-1 Acc.: 33.2% vs 30.1% | Improved |
Closest papers by meaning
- Vision‑Language‑Vision Auto‑Encoder: Scalable Knowledge Distillation from Diffusion Models (NeurIPS 2025 · similarity 0.88)
- DS-VLM: Diffusion Supervision Vision Language Model (ICML 2025 · similarity 0.87)
- Unified Generative and Discriminative Training for Multi-modal Large Language Models (NeurIPS 2024 · similarity 0.87)
- Beyond Words: Augmenting Discriminative Richness via Diffusions in Unsupervised Prompt Learning (CVPR 2025 · similarity 0.86)
- InstructCV: Instruction-Tuned Text-to-Image Diffusion Models as Vision Generalists (ICLR 2024 · similarity 0.86)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
