Research map / Image generation / Super-resolution
PassionSR: Post-Training Quantization with Adaptive Scale in One-Step Diffusion based Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PassionSR, a post-training quantization method for one-step diffusion super-resolution that simplifies OSEDiff to UNet-VAE and quantizes it to 8-bit and 6-bit using learnable boundary quantizer, learnable equivalent transformation, and two-stage distributed calibration.
Problem
One-step diffusion super-resolution models remain computationally and storage heavy for deployment, and existing quantization methods designed for multi-step diffusion degrade significantly on them.
Approach
Simplify the OSEDiff model to UNet and VAE by replacing the CLIP text branch with a constant empty-prompt embedding, then apply PTQ with learnable clipping boundaries (LBQ), learnable channel-wise scale/shift equivalent transformations (LET), and a two-stage calibration (DQC) that stabilizes training.
Key results
- At W8A8 on RealSR, PassionSR reaches PSNR 25.67 vs 25.27 for FP OSEDiff, while MaxMin, LSQ, Q-Diffusion and EfficientDM drop to 23.16, 15.39, 24.88 and 14.77 PSNR.
- PassionSR-UV at W8A8 reduces parameters by 81.77% (1,303M to 238M) and Ops by 76.56% (4,523G to 1,060G); at W6A6 it reduces parameters by 86.32% and Ops by 82.42%.
- Ablation at W6A6 on RealSR: adding LET raises PSNR from 23.15 to 25.40, and DQC cuts calibration time from 3.87 to 1.07 hours and GPU memory from 40 GB to 28 GB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add LET to LBQ (ablation) | PassionSR W6A6 · RealSR | PSNR: 23.15 -> 25.40 | Helped |
| add LET to LBQ (ablation) | PassionSR W6A6 · RealSR | calibration time: 2.66 -> 3.87 | Hurt |
| add DQC to LBQ+LET calibration (ablation) | PassionSR W6A6 · RealSR | calibration time: 3.87 -> 1.07 | Helped |
| add DQC to LBQ+LET calibration (ablation) | PassionSR W6A6 · RealSR | GPU memory: 40 -> 28 | Helped |
| add DQC to LBQ+LET calibration (ablation) | PassionSR W6A6 · RealSR | PSNR: 25.40 -> 24.41 | Hurt |
| remove DAPE and replace the CLIPEncoder branch with a constant empty-prompt embedding (ablation) | OSEDiff · RealSR | PSNR: 25.27 -> 25.39 | Helped |
| remove DAPE and replace the CLIPEncoder branch with a constant empty-prompt embedding (ablation) | OSEDiff · DRealSR | PSNR: 25.57 -> 26.70 | Helped |
| remove DAPE and replace the CLIPEncoder branch with a constant empty-prompt embedding (ablation) | OSEDiff · DIV2K val | PSNR: 24.95 -> 25.16 | Helped |
| quantize UNet and VAE with PassionSR W8A8 instead of using full precision OSEDiff | OSEDiff · RealSR | PSNR: 25.27 -> 25.67 | Helped |
| quantize UNet and VAE with PassionSR W8A8 instead of using full precision OSEDiff | OSEDiff · DRealSR | PSNR: 25.57 -> 27.41 | Helped |
| quantize UNet and VAE with PassionSR W8A8 instead of using full precision OSEDiff | OSEDiff · DIV2K val | PSNR: 24.95 -> 25.11 | Helped |
Surprising findings
- Quantized outputs from some baselines obtain high non-reference IQA scores despite visible quantization noise, so NR-IQA metrics can be misleading.
- Removing the CLIP text branch (PassionSR-FP) gives nearly identical quality to OSEDiff, and PassionSR at 8-bit sometimes exceeds the full-precision model on some metrics.
Limitations
- Evaluated only on UNet-VAE quantization with weights and activations at 8 and 6 bits; lower bit-widths not explored.
- Calibration requires a small set of 500 DIV2K crops and per-layer learned parameters, and training still takes hours of GPU time (LET stage about 3.87 h).
Experimental setup
Datasets: DIV2K train RealSR DRealSR DIV2K val
Benchmarks: RealSR DRealSR DIV2K val
Metrics: PSNR SSIM LPIPS DISTS NIQE MUSIQ MANIQA CLIP-IQA
Baselines: MaxMin LSQ Q-Diffusion EfficientDM OSEDiff DiffBIR
Builds on: OSEDiff SmoothQuant OmniQuant LSQ
Closest papers by meaning
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.89)
- EfficientDM: Efficient Quantization-Aware Fine-Tuning of Low-Bit Diffusion Models (ICLR 2024 · similarity 0.89)
- One-Step Effective Diffusion Network for Real-World Image Super-Resolution (NeurIPS 2024 · similarity 0.88)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.88)
- 2DQuant: Low-bit Post-Training Quantization for Image Super-Resolution (NeurIPS 2024 · similarity 0.88)
- Towards Accurate Post-training Quantization for Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.