Research map / Image generation / Fast diffusion sampling
Enhancing Consistency-Based Image Generation via Adversarialy-Trained Classification and Energy-Based Discrimination
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a post-processing method that refines Consistency-model images with PGD guided by a joint adversarially trained classifier-discriminator (energy-based view), improving FID on ImageNet 64x64.
Problem
Consistency models generate images in one or two steps but lag diffusion models in sample quality; the goal is to improve perceptual quality of their outputs after generation.
Approach
A single robust ResNet classifier is trained adversarially with both a CE classification loss and a BCE discrimination loss on real vs. synthetic images, so its logits act as an energy function; PGD steps guided by CE minus BCE (or a BCE-targeted variant) then push synthesized images toward the data manifold and the target class.
Key results
- On ImageNet 64x64, Wide-RN robust joint model with SGLD reduces CT 1-step FID from 13.00 to 8.71 and CD 2-step FID from 4.69 to 3.58.
- The joint model outperforms the robust-classifier-only BIGROC baseline (e.g., CT 1-step FID 10.71 vs 9.60 for RN50 joint with PGD).
- Trained only on Consistency-generated images, the joint model also improves ADM 64x64 FID from 2.61 to 2.08 and ADM-G 256x256 FID from 4.58 to 3.17.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Refine Consistency Training and Consistency Distillation samples with a robust joint classifier-discriminator instead of the unrefined generator | Consistency models · ImageNet 64×64, 1- and 2-step generation | FID: 13.00 -> 8.83; 11.12 -> 7.98; 6.20 -> 4.84; 4.69 -> 3.78 | Helped |
| Use the proposed sampling loss instead of Cross-Entropy loss (ablation) | Joint classifier-discriminator refinement · ImageNet 64×64, Consistency Training and Consistency Distillation | FID: 10.78 -> 9.60; 10.26 -> 8.51; 5.51 -> 4.95; 4.17 -> 3.84 | Helped |
| Use Binary Cross-Entropy loss instead of Cross-Entropy loss (ablation) | Joint classifier-discriminator refinement · ImageNet 64×64, Consistency Training and Consistency Distillation | FID: 10.78 -> 9.75; 10.26 -> 8.59; 5.51 -> 5.00; 4.17 -> 3.89 | Helped |
| Use CE minus BCE sampling loss instead of Cross-Entropy loss (ablation) | Joint classifier-discriminator refinement · ImageNet 64×64, Consistency Training and Consistency Distillation | FID: 10.78 -> 9.71; 10.26 -> 8.62; 5.51 -> 5.10; 4.17 -> 4.97 | Mixed |
| Replace PGD sampling with SGLD (ablation) | Wide-RN joint refinement · ImageNet 64×64, Consistency Training and Consistency Distillation | FID: 8.83 -> 8.71; 7.98 -> 7.64; 4.84 -> 4.65; 3.78 -> 3.58 | Helped |
Surprising findings
- SGLD sampling at inference outperforms PGD, although SGLD was worse than PGD when used during training.
- The joint model, trained only on Consistency-generated images, generalizes to images from BigGAN, ADM and IDDPM at other resolutions without retraining.
Limitations
- Uses small architectures (RN50 and Wide-RN-50-2), which may constrain performance.
- Training relies only on Consistency-generated images; adding other generators might help further.
Experimental setup
Datasets: ImageNet 64x64
Benchmarks: ImageNet 64x64 ImageNet 128x128 (ADM-G, BigGAN) ImageNet 256x256 (ADM-G, BigGAN)
Metrics: FID Inception Score Precision Recall
Baselines: BIGROC (robust classifier boosting) Consistency Models (CT and CD) without boosting Adversarial Consistency Training (ACT)
Builds on: BIGROC Consistency Models Projected Gradient Descent Energy-Based Models Perceptually Aligned Gradients
Closest papers by meaning
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.90)
- ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models (CVPR 2024 · similarity 0.90)
- Consistency-GAN: Training GANs with Consistency Model (AAAI 2024 · similarity 0.90)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.88)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.87)
- Robust Representation Consistency Model via Contrastive Denoising (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.