Research map / Image generation / Fast diffusion sampling
An Efficient Framework for Enhancing Discriminative Models via Diffusion Techniques
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DBMEF, a training-free plug-and-play framework that uses a confidence protector hypothesis test to select uncertain predictions from a discriminative classifier and reclassifies their top-k labels with Stable Diffusion conditional denoising errors (positive/negative text conditions, voting).
Problem
Discriminative image classifiers lack a mechanism to re-evaluate uncertain predictions, while diffusion-based generative classifiers are slow and underperform discriminative models on ImageNet.
Approach
Combine a discriminative model's top-k predictions with a diffusion classifier applied only to low-confidence samples, selected by a confidence protector (significance-level threshold on max softmax from correctly classified training samples), and score candidate labels using a negative-control-weighted combination of positive and negative text-conditioned noise predictions.
Key results
- DBMEF improves ResNet-50 on ImageNet from 76.15% to 77.66% (+1.51%) across 17 discriminative models, with gains from 0.19% to 3.27%.
- On ImageNet-A, ResNet-50 improves from 0.00% to 3.02% under distribution shift.
- Inference at 30 timesteps takes about 1% of the time of pure diffusion classifiers (ResNet50 0.252 vs 0.209 h without DBMEF on ImageNet val).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DBMEF with confidence protection, combined positive and negative conditions, and voting | ResNet50 · ImageNet-1k val | top-1 accuracy: 76.15% -> 77.66% | Helped |
| Add DBMEF with confidence protection, combined positive and negative conditions, and voting | ViTh-CLIP · ImageNet-1k val | top-1 accuracy: 88.59% -> 88.78% | Helped |
| Add DBMEF | ResNet50 · ImageNet-A | accuracy: 0.00% -> 3.02% | Helped |
| Add DBMEF | ResNet50 · ImageNet-S, ImageNet-V2, ImageNet-E (background), and ImageNet-E (position) | accuracy: 24.08% -> 25.82%; 72.30% -> 72.98%; 62.22% -> 63.35%; 65.17% -> 65.31% | Helped |
| Add DBMEF | ResNet50 · CIFAR-10 test and CIFAR-100 test | accuracy: 96.29% -> 96.30%; 83.85% -> 84.46% | Helped |
| Add confidence protection rather than reclassifying all samples (ablation) | ResNet50 · ImageNet-1k val | top-1 accuracy: 64.39% -> 77.40% | Helped |
| Add positive and negative text-condition combination rather than positive text only (ablation) | ResNet50 with confidence protection · ImageNet-1k val | top-1 accuracy: 77.22% -> 77.40% | Helped |
| Add voting over five combined-condition predictions (ablation) | ResNet50 with DBMEF · ImageNet-1k val | top-1 accuracy: 77.40% -> 77.66% | Helped |
| Use negative text only instead of combined positive and negative conditions (ablation) | ResNet50 with confidence protection · ImageNet-1k val | top-1 accuracy: 77.40% -> 76.68% | Hurt |
| Increase timesteps from 5 to 1000 (ablation) | ResNet50 with DBMEF · ImageNet-1k val | top-1 accuracy: 76.18% -> 77.60% | Helped |
| Set Prot to 0.90 rather than 0.95 (ablation) | ResNet50 with DBMEF · ImageNet-1k val | top-1 accuracy: 77.40% -> 76.70% | Hurt |
| Skip reclassification by setting Prot to 1.00 (ablation) | ResNet50 with DBMEF · ImageNet-1k val | top-1 accuracy: 77.40% -> 76.15% | Hurt |
Surprising findings
- Reclassifying all images with the diffusion model without the confidence protector drops accuracy well below the discriminative baseline (e.g., ResNet-50 to 64.39%).
- Using a discriminative model (ResNet101, ViT-CLIP) as the second stage in place of the diffusion model yields negative gains, while SD gives positive gains.
Limitations
- Gains are modest on strong models (0.19%-0.8% on ViTh-CLIP and ViTb-MAE).
- Hyperparameters such as Prot and timesteps were not tuned per model.
Experimental setup
Datasets: ImageNet ImageNet-A ImageNet-V2 ImageNet-S ImageNet-E CIFAR-10 CIFAR-100
Benchmarks: ImageNet ImageNet-A ImageNet-V2 ImageNet-S ImageNet-E CIFAR-10 CIFAR-100
Metrics: top-1 accuracy
Baselines: w/o p (unprotected diffusion reclassification) Stable Diffusion classifier (Li et al.) DiT-XL/2 diffusion classifier
Builds on: Stable Diffusion diffusion classifier (zero-shot density estimation) classifier-free guidance
Closest papers by meaning
- Not All Diffusion Model Activations Have Been Evaluated as Discriminative Features (NeurIPS 2024 · similarity 0.89)
- Diffusion Classifiers Understand Compositionality, but Conditions Apply (NeurIPS 2025 · similarity 0.87)
- ADDP: Learning General Representations for Image Recognition and Generation with Alternating Denoising Diffusion Process (ICLR 2024 · similarity 0.86)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.86)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.86)
- Discriminative Score Function: Turning Pretrained Models into Functional Generative Priors (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.