Research map / Image generation / Fast diffusion sampling
Early Estimation of Language to Latent Alignment in Diffusion Models
Vasco Ramos, Regev Cohen, Idan Szpektor, Joao Magalhaes
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune CLIP image encoder on noisy latents (NoisyCLIP) | CLIP L14 · Noisy-Conceptual-Captions latent 21-30 | Factual consistency R@1: 0.145 -> 0.506 | Improved |
| fine-tune CLIP image encoder on noisy latents (NoisyCLIP) | CLIP L14 · Noisy-GenAI-Bench latent 21-30 Best-of-N | VQAScore: 0.657 -> 0.709 | Improved |
| noise-aware fine-tuning of SigLIP (NoisySigLIP) | SigLIP L16 · Noisy-GenAI-Bench latent 21-30 Best-of-N | VQAScore: 0.652 -> 0.718 | Improved |
| noise-aware fine-tuning of SigLIP (NoisySigLIP) | SigLIP L16 · Noisy-Conceptual-Captions latent 21-30 | Factual consistency R@1: 0.147 -> 0.194 | Improved |
| early-stopping BoN at 150 steps instead of 300-step CLIP selection | NoisyCLIP · GenAI-Bench basic, best-of-6 | VQAScore: 0.85 -> 2% drop at 50% cost | Mixed |
| add NoisyCLIP selection at latent 20 for BoN | NoisyCLIP · GenAI-Bench BoN at latent 20 | VQAScore: up to 2% | Improved |
| train image encoder only (add image encoder training) (ablation) | CLIP · GenAI Basic, latent 21-30 | VQAScore: 0.821 -> 0.836 | Improved |
| train image encoder only (add image encoder training) (ablation) | CLIP · GenAI Advanced, latent 21-30 | VQAScore: 0.633 -> 0.646 | Improved |
| also train text encoder in addition to image encoder (ablation) | NoisyCLIP · GenAI Advanced, image dimension | VQAScore: 0.664 -> 0.657 | Hurt |
| also train text encoder in addition to image encoder (ablation) | NoisyCLIP · GenAI Basic, latent 21-30 | VQAScore: 0.836 -> 0.835 | No clear effect |
| scale encoder from ViT-B/16 to ViT-L/14 (ablation) | NoisyCLIP · GenAI Basic, latent 21-30 | VQAScore: 0.829 -> 0.836 | Improved |
| scale encoder from ViT-B/16 to ViT-L/14 (ablation) | NoisyCLIP · GenAI Advanced, image dimension | VQAScore: 0.652 -> 0.664 | Improved |
Closest papers by meaning
- Noise Diffusion for Enhancing Semantic Faithfulness in Text-to-Image Synthesis (CVPR 2025 · similarity 0.90)
- Inference-Time Text-to-Video Alignment with Diffusion Latent Beam Search (NeurIPS 2025 · similarity 0.87)
- Uncertainty Quantification for Large Language Diffusion Models (NeurIPS 2026 · similarity 0.87)
- Information Theoretic Text-to-Image Alignment (ICLR 2025 · similarity 0.87)
- Denoising Time Matters: Diverse Generation in Diffusion Language Models (NeurIPS 2026 · similarity 0.87)
- Towards General Modality Translation with Contrastive and Predictive Latent Diffusion Bridge (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
