Research map / Vision-language and multimodal / CLIP and zero-shot
Text-to-Image Diffusion Models are Zero Shot Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use diffusion denoising error conditioned on class text as the zero-shot class score | Imagen and Stable Diffusion · Image classification benchmarks | accuracy | Mixed |
| Use the hand-designed timestep weighting exp(-7t) instead of the Simple or VDM weighting (ablation) | Diffusion generative classifier · Caltech101 | accuracy: Imagen: 56.1 -> 68.9 (Simple), 62.0 -> 68.9 (VDM); Stable Diffusion: 71.4 -> 73.0 (Simple), 71.9 -> 73.0 (VDM) | Helped |
| Use learned timestep weights instead of heuristic weights (ablation) | Diffusion generative classifier · Caltech101 | accuracy: Imagen: 68.9 -> 70.2; Stable Diffusion: 73.0 -> 73.1 | Helped |
| Use shared noise across candidate classes (ablation) | Diffusion generative classifier · CIFAR-100 | sample efficiency: roughly 100x | Helped |
| Prune implausible candidate classes early (ablation) | Diffusion generative classifier · CIFAR-100 | sample efficiency: 8-10x | Helped |
| Combine shared noise and candidate pruning rather than naive scoring (ablation) | Diffusion generative classifier · CIFAR-100 | compute reduction: up to 1000x | Helped |
| Downsample ImageNet inputs to 32x32 or 64x64 instead of the full input resolution (ablation) | Stable Diffusion · ImageNet | accuracy: 61.9% -> 15.5%; 61.9% -> 34.6% | Hurt |
| Use the 64x64-to-256x256 super-resolution model as classifier instead of the 64x64 Imagen model (ablation) | Imagen · ImageNet | accuracy: 16.1% | Hurt |
| Combine low-resolution and super-resolution Imagen scores (ablation) | Imagen · ImageNet classification | accuracy | No clear effect |
| Replace the single prompt with alternative CLIP prompt templates (ablation) | Diffusion generative classifier · CIFAR-100 | accuracy: within 1.5% | No clear effect |
| Add the second object's attribute to form a pair prompt rather than test attribute-to-object binding (ablation) | Imagen · Synthetic CLEVR attribute tasks | accuracy: Shape,Color: 100; Shape|Color: 66; Color|Shape: 73 | Helped |
| Use Stable Diffusion for positional attribute binding (ablation) | Stable Diffusion · Synthetic CLEVR attribute tasks | accuracy: Shape|Position: 50; Position|Shape: 50; Color|Position: 51; Position|Color: 49 | No clear effect |
Closest papers by meaning
- Diffusion Classifiers Understand Compositionality, but Conditions Apply (NeurIPS 2025 · similarity 0.92)
- Diffusion Bridge: Leveraging Diffusion Model to Reduce the Modality Gap Between Text and Vision for Zero-Shot Image Captioning (CVPR 2025 · similarity 0.90)
- Magnet: We Never Know How Text-to-Image Diffusion Models Work, Until We Learn How Vision-Language Models Function (NeurIPS 2024 · similarity 0.90)
- Diffusion Self-Distillation for Zero-Shot Customized Image Generation (CVPR 2025 · similarity 0.90)
- Dual Diffusion for Unified Image Generation and Understanding (CVPR 2025 · similarity 0.90)
- ORCA: Hunting Compositional Failures in Text-to-Image Diffusion (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.