Research map / Label-efficient and robust learning / Recognition and long-tail
Generative Classifiers Avoid Shortcut Solutions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that generative classifiers (diffusion-based for images, autoregressive for text) trained from scratch avoid shortcut learning and outperform discriminative baselines on five distribution shift benchmarks, with a Gaussian LDA analysis explaining the inductive bias.
Problem
Discriminative classifiers learn spurious shortcut features that work in-distribution but fail under distribution shift, and existing mitigations need extra hyperparameters, stages, or knowledge of the spurious correlations.
Approach
Use class-conditional generative models (latent diffusion for images, Llama-style autoregressive models with class tokens for text) as classifiers via Bayes' rule, so the objective must model all features, core and spurious, which yields consistent learning signal and an inductive bias toward consistently predictive features.
Key results
- Generative classifiers reach 79.4% worst-group accuracy on Waterbirds vs 32.2% for ERM and 90.8% OOD on Camelyon vs 78.3% for ERM.
- Generative classifiers achieve 69.4% worst-group accuracy on CelebA (vs 50.5% ERM) and 61.4% on CivilComments (vs 53.3% ERM), while ID accuracy on CivilComments is lower (79.8 vs 90.6).
- On FMoW, the generative classifier gets 35.8% OOD worst-group accuracy vs 27.5% for ERM, and it shows effective robustness on CelebA, CivilComments and possibly Camelyon.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use generative classification instead of discriminative baselines | Waterbirds · Waterbirds WG | accuracy: 32.9 -> 79.4 | Helped |
| Use generative classification instead of discriminative baselines | CelebA · CelebA WG | accuracy: 68.9 -> 69.4 | Helped |
| Use generative classification instead of discriminative baselines | Camelyon · Camelyon OOD | accuracy: 78.3 -> 90.8 | Helped |
| Use generative classification instead of discriminative baselines | FMoW · FMoW OOD WG | accuracy: 31.8 -> 35.8 | Helped |
| Use generative classification instead of discriminative baselines | CivilComments · CivilComments WG | accuracy: 58.1 -> 61.4 | Helped |
| Use generative classification instead of discriminative baselines | CelebA · CelebA ID | accuracy: 94.1 -> 91.2 | Hurt |
| Use generative classification instead of discriminative baselines | CivilComments · CivilComments ID | accuracy: 90.6 -> 79.8 | Hurt |
| Replace the pre-trained VAE embedding with PCA patch embeddings (ablation) | Generative classifier · Waterbirds | WG accuracy: 79.4 -> 61.7 | Hurt |
| Replace the pre-trained VAE embedding with PCA patch embeddings (ablation) | Generative classifier · CelebA | WG accuracy: 69.4 -> 71.1 | Helped |
| Replace the pre-trained VAE embedding with PCA patch embeddings (ablation) | Generative classifier · Camelyon | OOD accuracy: 90.8 -> 93.8 | Helped |
| Add an unconditional text modeling objective p(x) (ablation) | CivilComments generative classifier · CivilComments | classification performance | No clear effect |
| Increase generative model size (ablation) | Generative classifier · FMoW and CivilComments | accuracy | Helped |
Surprising findings
- Generative classifiers can classify well even when generated samples have visible artifacts, since only relative likelihoods across classes matter.
- Pretraining a discriminative ResNet-50 on ImageNet and finetuning helps but does not close the effective robustness gap with the from-scratch generative classifier.
Limitations
- Diffusion-based generative classifier inference is expensive, requiring many noise samples per class.
- Unclear how to incorporate complex augmentations such as Mixup into generative classifiers.
- Generative classifiers are worse in-distribution on CivilComments and CelebA shows no ID gain, and results on Camelyon are noisy.
Experimental setup
Datasets: Waterbirds CelebA Camelyon17-WILDS FMoW CivilComments BREEDS Living-17 BREEDS Entity-30 ImageNet-1k
Benchmarks: Waterbirds CelebA Camelyon17-WILDS FMoW CivilComments BREEDS Living-17 BREEDS Entity-30
Metrics: worst-group accuracy out-of-distribution accuracy in-distribution accuracy class-balanced accuracy
Baselines: ERM LfF JTT DFR/RWY
Builds on: Diffusion Classifier latent diffusion models Learning from Failure Just Train Twice Deep Feature Reweighting
Closest papers by meaning
- Intriguing Properties of Generative Classifiers (ICLR 2024 · similarity 0.90)
- Generative Cross-Entropy: A Strictly Proper Loss for Data-Efficient Classification (NeurIPS 2026 · similarity 0.88)
- Synthetic Tabular Data Generation for Imbalanced Classification: The Surprising Effectiveness of an Overlap Class (AAAI 2025 · similarity 0.86)
- A Likelihood Based Approach to Distribution Regression Using Conditional Deep Generative Models (ICML 2025 · similarity 0.86)
- Are Images Indistinguishable to Humans Also Indistinguishable to Classifiers? (CVPR 2025 · similarity 0.86)
- Let Samples Speak: Mitigating Spurious Correlation by Exploiting the Clusterness of Samples (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.