Research map / Image generation / Super-resolution
Effective Diffusion Transformer Architecture for Image Super-Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiT-SR, a U-shaped diffusion transformer with isotropic transformer blocks and a frequency-domain time-step conditioning module (AdaFM) for training-from-scratch image super-resolution.
Problem
Diffusion-based super-resolution methods trained from scratch lag far behind prior-based methods built on pretrained Stable Diffusion, and AdaLN time conditioning is poorly suited to SR's frequency-dependent denoising.
Approach
Combine a U-shaped multi-scale encoder-decoder with isotropic transformer blocks sharing one widened channel dimension across stages (allocating more compute to high-resolution layers), and replace AdaLN with AdaFM, which modulates features by a time-dependent scale matrix in the FFT frequency domain.
Key results
- On RealSR, Ours-15 (61M params) reaches CLIPIQA 0.7161 and MUSIQ 65.83, beating ResShift-15 (119M, CLIPIQA 0.6028) and LDM-100 among training-from-scratch methods.
- Replacing AdaLN with AdaFM in the same architecture reduces denoiser parameters from 100.64M to 60.79M while improving RealSR CLIPIQA from 0.700 to 0.716.
- The proposed U-shaped isotropic design uses 100.64M parameters vs 264.39M for the U-shaped DiT, with FLOPs reduced by 24% (122.87G to 93.11G) and better CLIPIQA/MUSIQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiT-SR instead of ResShift-15 | Training-from-scratch SR methods · RealSR | CLIPIQA: 0.6028 -> 0.7161 | Helped |
| Use DiT-SR instead of ResShift-15 | Training-from-scratch SR methods · LSDIR-Test | PSNR: 23.83 -> 23.60 | Hurt |
| Use DiT-SR instead of ResShift-15 | Training-from-scratch SR methods · LSDIR-Test | LPIPS: 0.247 -> 0.244 | Helped |
| Replace AdaLN with AdaFM (ablation) | DiT-SR · RealSR | CLIPIQA: 0.700 -> 0.716 | Helped |
| Replace AdaLN with AdaFM (ablation) | DiT-SR · RealSet65 | MUSIQ: 67.634 -> 66.741 | Hurt |
| Add U-shaped structure to isotropic DiT (ablation) | DiT-SR with AdaLN · RealSR | CLIPIQA: 0.655 -> 0.700 | Helped |
| Add isotropic block design to U-shaped DiT (ablation) | DiT-SR with AdaLN · RealSR | CLIPIQA: 0.688 -> 0.700 | Helped |
| Reduce transformer blocks per stage from 6 to 4 (ablation) | U-shaped DiT · RealSR | CLIPIQA: 0.688 -> 0.671 | Hurt |
| Reduce base channel dimension from 160 to 144 (ablation) | U-shaped DiT · RealSR | CLIPIQA: 0.688 -> 0.682 | Hurt |
| Reduce depth and width and remove the deepest layer (ablation) | DiT-SR · RealSR | CLIPIQA: 0.7161 -> 0.6670 | Hurt |
| Distill from 15 denoising steps to one step (ablation) | Ours-Lite · RealSR | CLIPIQA: 0.6670 -> 0.6993 | Helped |
| Distill from 15 denoising steps to one step (ablation) | Ours-Lite · RealSR | MANIQA: 0.4565 -> 0.4262 | Hurt |
Surprising findings
- The proposed architecture outperforms the U-shaped DiT while using only about 48% of its parameters.
- Reallocating channels to high-resolution stages improves performance even with a 62% parameter reduction relative to the U-shaped DiT.
Limitations
- Still does not fully match the performance of prior-based SD methods, per the authors.
- SR models show less scalability than text-to-image models due to task differences and limited data.
Experimental setup
Datasets: LSDIR DIV2K DIV8K OutdoorSceneTraining Flickr2K FFHQ RealSR RealSet65 CelebA-HQ LFW WebPhoto WIDER
Benchmarks: LSDIR-Test RealSR RealSet65 CelebA-Test LFW WebPhoto WIDER
Metrics: PSNR LPIPS CLIPIQA MUSIQ MANIQA IDS LMD FID
Baselines: RealSR-JPEG BSRGAN RealESRGAN SwinIR StableSR DiffBIR PASD SeeSR LDM ResShift
Builds on: DiT ResShift AdaLN SwinIR LDM
Closest papers by meaning
- SinSR: Diffusion-Based Image Super-Resolution in a Single Step (CVPR 2024 · similarity 0.91)
- Unleashing the Power of One-Step Diffusion based Image Super-Resolution via a Large-Scale Diffusion Discriminator (NeurIPS 2025 · similarity 0.91)
- Solving Diffusion ODEs with Optimal Boundary Conditions for Better Image Super-Resolution (ICLR 2024 · similarity 0.90)
- Arbitrary-steps Image Super-resolution via Diffusion Inversion (CVPR 2025 · similarity 0.90)
- ResDiff: Combining CNN and Diffusion Model for Image Super-resolution (AAAI 2024 · similarity 0.90)
- One Diffusion Step to Real-World Super-Resolution via Flow Trajectory Distillation (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.