Research map / Image generation / Autoregressive and latent generation
Intriguing Properties of Generative Classifiers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns zero-shot text-to-image models (Imagen, Stable Diffusion, Parti) into classifiers via diffusion/likelihood scoring and benchmarks their shape bias, OOD accuracy, error consistency with humans, and illusion understanding against discriminative models and human data.
Problem
Whether discriminative or generative paradigms better model human object recognition, especially out-of-distribution robustness and human-like decision patterns.
Approach
Extract classification decisions from text-to-image generative models by scoring each class-conditioned reconstruction (diffusion ELBO approximation via L2 loss, or exact likelihood for autoregressive Parti), and evaluate them zero-shot on the model-vs-human benchmark.
Key results
- Imagen achieves 99% shape bias, Stable Diffusion 93%, Parti 92%, versus 96% average for humans and 87% for ViT-22B-384.
- Imagen reaches 0.71 OOD accuracy and 0.31 error consistency, the highest error consistency among models in the benchmark (ViT-22B-384: 0.26).
- A ResNet-50 trained with diffusion-style noise augmentation raises shape bias from 21% to 78%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a generative classifier instead of a discriminative model | Imagen vs. ViT-22B-384 · Cue-conflict images | shape bias: 87% -> 99% | Helped |
| use a generative classifier instead of a discriminative model | Imagen vs. ViT-22B-384 · 17 OOD datasets | OOD accuracy: 0.80 -> 0.71 | Hurt |
| use Imagen as the classifier | Imagen vs. ViT-22B-384 · 17 OOD datasets | error consistency: 0.26 -> 0.31 | Helped |
| add diffusion-style noise during training (ablation) | ResNet-50 · Cue-conflict images | shape bias: 21% -> 57% | Helped |
| add diffusion-style noise during training and evaluation (ablation) | ResNet-50 · Cue-conflict images | shape bias: 21% -> 78% | Helped |
| add diffusion-style noise during training and evaluation (ablation) | ResNet-50 · 17 OOD datasets | OOD accuracy: 0.56 -> 0.43 | Hurt |
| use 80 prompts instead of 1 prompt (ablation) | CLIP · Cue-conflict images | shape bias: 80% -> 57% | Hurt |
| use 80 prompts instead of 1 prompt (ablation) | CLIP · 17 OOD datasets | OOD accuracy: 0.55 -> 0.71 | Helped |
| use 80 prompts instead of 1 prompt (ablation) | CLIP · 17 OOD datasets | error consistency: 0.26 -> 0.28 | Helped |
| use the 80-prompt CLIP ensemble (ablation) | CLIP · ImageNet validation | accuracy | Hurt |
Surprising findings
- Zero-shot generative classifiers show human-level shape bias despite not being trained for classification.
- Generative classifiers struggle to generate rotated images, mirroring their poor accuracy on rotation distortions.
Limitations
- Generative classifiers are computationally slow, requiring per-class likelihood evaluation.
- Compared models differ in architecture, data, resolution and training, so comparisons are confounded.
- Muse could not be used as a working classifier with the L2 reconstruction approach.
Experimental setup
Datasets: model-vs-human toolbox 17 OOD datasets ImageNet-1K cue-conflict images
Benchmarks: model-vs-human 17 OOD datasets cue-conflict shape bias error consistency with human observers
Metrics: shape bias OOD accuracy error consistency (Cohen's kappa) accuracy difference observed consistency
Baselines: ViT-22B CLIP ViT-L/14 (1 prompt and 80 prompts) ResNet-50 variants BiT-M SimCLR Noisy Student ENetL2
Builds on: Imagen Stable Diffusion Parti model-vs-human toolbox diffusion classifier (Li et al. 2023)
Closest papers by meaning
- Generative Classifiers Avoid Shortcut Solutions (ICLR 2025 · similarity 0.90)
- Not Too Generative, Not Too Discriminative: The Human Alignment Sweet Spot (NeurIPS 2026 · similarity 0.87)
- Generative Region-Language Pretraining for Open-Ended Object Detection (CVPR 2024 · similarity 0.87)
- Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors (NeurIPS 2026 · similarity 0.87)
- L-WISE: Boosting Human Visual Category Learning Through Model-Based Image Selection and Enhancement (ICLR 2025 · similarity 0.86)
- Hidden in Plain Sight: Evaluating Abstract Shape Recognition in Vision-Language Models (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.