Research map / Image generation / Super-resolution
Adversarial Diffusion Compression for Real-World Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Compresses the one-step SD-based Real-ISR model OSEDiff into a smaller diffusion-GAN (AdcSR) by removing modules (VAE encoder, prompt extractor, text/time modules), channel-pruning the UNet and VAE decoder, and distilling in feature space with adversarial loss.
Problem
Real-world image super-resolution with Stable Diffusion-based one-step models such as OSEDiff and S3Diff remains too computationally expensive (parameters, MACs, latency) for practical and edge deployment.
Approach
Adversarial Diffusion Compression: remove redundant modules (VAE encoder, prompt extractor, text encoder, cross-attention, time embeddings), feed PixelUnshuffled LR input directly to the UNet, connect UNet features directly to the VAE decoder, prune channels, pretrain the pruned VAE decoder, then distill OSEDiff's features with an L1 loss plus an SD-UNet-based adversarial loss using GT features.
Key results
- AdcSR on DRealSR achieves 0.03 s inference, 496G MACs and 456M parameters, versus OSEDiff's 0.11 s, 2265G MACs and 1775M parameters (3.7x speedup, 78% fewer MACs, 74% fewer parameters).
- AdcSR reaches 28.10 PSNR, 0.7726 SSIM, 0.3046 LPIPS on DRealSR, with MUSIQ 66.26 and CLIPIQA 0.7049, competitive with OSEDiff (27.92 PSNR, 0.2968 LPIPS) and S3Diff.
- Stated speedup of up to 9.3x over S3Diff; runs at 34.79 FPS on an A100 GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Compress OSEDiff with ADC structural removal, channel pruning, and adversarial distillation | OSEDiff · DRealSR | inference time (s): 0.11 -> 0.03 | Helped |
| Compress OSEDiff with ADC structural removal, channel pruning, and adversarial distillation | OSEDiff · DRealSR | MACs (G): 2265 -> 496 | Helped |
| Eliminate the VAE encoder (ablation) | AdcSR · DRealSR | PSNR: 27.97 -> 28.10 | Helped |
| Eliminate the VAE encoder (ablation) | AdcSR · DRealSR | LPIPS: 0.3077 -> 0.3046 | Helped |
| Optimize the connection between the denoising UNet and VAE decoder (ablation) | AdcSR · DRealSR | FID: 140.09 -> 134.05 | Helped |
| Remove the prompt extractor and text/CA modules (ablation) | AdcSR · RealSR | DISTS: 0.2116 -> 0.2130 | Hurt |
| Remove time embeddings in addition to prompt extractor and text/CA modules (ablation) | AdcSR · RealSR | DISTS: 0.2130 -> 0.2129 | No clear effect |
| Prune 25% of UNet channels while pruning 50% of VAE decoder channels (ablation) | AdcSR · RealSR | LPIPS: 0.2839 -> 0.2885 | Hurt |
| Use more aggressive pruning: 50% UNet and 50% VAE decoder channels (ablation) | AdcSR · RealSR | FID: 118.41 -> 125.10 | Hurt |
| Replace teacher feature distillation with GT feature matching (ablation) | AdcSR · RealSR | NIQE: 5.35 -> 8.59 | Hurt |
| Distill in image space instead of the feature space (ablation) | AdcSR · RealSR | PSNR: 25.47 -> 25.80 | Hurt |
| Remove the adversarial loss (ablation) | AdcSR · DRealSR | LPIPS: 0.3046 -> 0.3261 | Hurt |
Surprising findings
- Removing transformer-style text prompts and cross-attention layers in the UNet had only minor quality cost (DISTS increase of 0.0014) while cutting parameters by 64%.
- Removing UNet/decoder depth (block removal) noticeably degraded one-step diffusion networks, so channel pruning was chosen instead.
Limitations
- Struggles to recover fine textures and heavily degraded details (shown in failure examples).
- Only the OSEDiff instantiation of ADC is studied; extension to other SD-based methods is left for future work.
- Efficiency numbers are measured on a single NVIDIA A100 GPU.
Experimental setup
Datasets: OpenImage LAION-Face LAION-Aesthetic LSDIR DIV2K-Val RealSR DRealSR
Benchmarks: DIV2K-Val RealSR DRealSR
Metrics: PSNR SSIM LPIPS DISTS FID NIQE MUSIQ MANIQA CLIPIQA inference time
Baselines: StableSR DiffBIR SeeSR PASD ResShift SinSR OSEDiff S3Diff
Builds on: OSEDiff SD-Turbo Stable Diffusion Diffusion2GAN BK-SDM
Closest papers by meaning
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.91)
- Noise-Started One-Step Image Super-Resolution via LR-Conditioned SplitMeanFlow and GAN Refinement (NeurIPS 2026 · similarity 0.91)
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.89)
- ResDiff: Combining CNN and Diffusion Model for Image Super-resolution (AAAI 2024 · similarity 0.88)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.88)
- Arbitrary-Scale Image Generation and Upsampling using Latent Diffusion Model and Implicit Neural Decoder (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.