Research map / Image generation / Fast diffusion sampling
ACT-Diffusion: Efficient Adversarial Consistency Training for One-step Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adversarial Consistency Training (ACT), which adds a timestep-conditioned discriminator to consistency training to directly minimize JS divergence per timestep, yielding better one-step image generation with much smaller batch sizes and compute.
Problem
Consistency training for one-step diffusion generation yields lower quality than distillation and needs very large batch sizes and models because its loss accumulates errors across timesteps.
Approach
Shows consistency training minimizes an upper bound on W-distance that accumulates past consistency losses, then adds a GAN-style discriminator (conditioned on timestep) to the consistency loss with an adversarial weight schedule, directly reducing JS divergence at each timestep; also adds adaptive differentiable augmentation driven by gradient penalty for small datasets.
Key results
- On CIFAR-10, ACT-Aug reaches FID 6.0 (batch size 80, 300K steps) versus consistency training's 8.7 (batch size 512, 800K steps).
- On ImageNet 64x64, ACT achieves FID 10.6 with batch size 320 versus CT's 13.0 at batch size 2048.
- On LSUN Cat 256x256, ACT achieves FID 13.0 with batch size 320 versus CT's 20.7 at batch size 2048.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adversarial consistency training to CT | DDPM · CIFAR10 | FID: 8.7 -> 6.0 | Helped |
| add adversarial consistency training to CT | DDPM · ImageNet 64×64 | FID: 13.0 -> 10.6 | Helped |
| add adversarial consistency training to CT | DDPM · LSUN Cat 256×256 | FID: 20.7 -> 13.0 | Helped |
| reduce CT batch size from 512 to 256 (ablation) | CT on DDPM · CIFAR10 | FID: 8.7 -> 10.4 | Hurt |
| reduce CT batch size from 512 to 128 (ablation) | CT on DDPM · CIFAR10 | FID: 8.7 -> 14.4 | Hurt |
| use DDPM-res discriminator with LeakyReLU and no timestep embedding (ablation) | ACT on CIFAR10 · CIFAR10, 50k steps | FID: 18.7 | Hurt |
| add timestep embedding to the DDPM-res discriminator with LeakyReLU (ablation) | ACT on CIFAR10 · CIFAR10, 50k steps | FID: 18.7 -> 11.5 | Helped |
| replace LeakyReLU with SiLU in the DDPM-res discriminator (ablation) | ACT on CIFAR10 · CIFAR10, 50k steps | FID: 11.5 -> 9.9 | Helped |
| remove residual connections between discriminator downsampling blocks (ablation) | ACT on CIFAR10 · CIFAR10, 150k steps | FID: 7.0 -> 6.5 | Helped |
| replace the DDPM discriminator with a StyleGAN2 discriminator (ablation) | ACT on CIFAR10 · CIFAR10, 150k steps | FID: 6.5 -> 9.5 | Hurt |
| replace the DDPM discriminator with a ProjectedGAN discriminator (ablation) | ACT on CIFAR10 · CIFAR10, 150k steps | FID: 6.5 -> 16.6 | Hurt |
| use an overly large adversarial weight (λN ≡ 0.3) (ablation) | ACT · ImageNet 64×64 | FID | Hurt |
Surprising findings
- Using SiLU rather than LeakyReLU as the discriminator activation improved convergence and final FID, contrary to GAN conventions.
- Overly large adversarial weight (lambda = 1, i.e., pure GAN) causes mode collapse, and lambda 0.3 on ImageNet collapses around 170k steps without proper weighting.
Limitations
- Interaction between consistency loss and adversarial loss can conflict; large adversarial weight causes training collapse or mode collapse.
- Data augmentation effectiveness on large datasets such as ImageNet 64x64 is unexplored.
- Only JS divergence is explored; other distance measures are left to future work.
Experimental setup
Datasets: CIFAR-10 ImageNet 64x64 LSUN Cat 256x256
Benchmarks: CIFAR-10 ImageNet 64x64 LSUN Cat 256x256
Metrics: FID Inception Score precision recall
Baselines: consistency training (CT) consistency distillation (CD) BigGAN StyleGAN2-ADA EDM DDPM DDIM Score SDE Progressive distillation ADM
Builds on: consistency models GANs StyleGAN2-ADA WGAN-GP DDPM
Closest papers by meaning
- Enhancing Consistency-Based Image Generation via Adversarialy-Trained Classification and Energy-Based Discrimination (NeurIPS 2024 · similarity 0.90)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.90)
- Truncated Consistency Models (ICLR 2025 · similarity 0.89)
- Improved Diffusion-based Generative Model with Better Adversarial Robustness (ICLR 2025 · similarity 0.89)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.89)
- Consistency-GAN: Training GANs with Consistency Model (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.