Research map / Architectures and efficiency / RNNs and normalization
ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection
Zhongzhan Huang, Pan Zhou, Shuicheng YAN, Liang Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add exponentially decaying long skip coefficient scaling (CS) | UNet and UViT · CIFAR10, ImageNet 64, MS-COCO, CelebA | FID: 3.17 -> 3.05; 3.77 -> 3.48; 7.32 -> 7.04; 2.87 -> 2.78 | Improved |
| Add learnable long skip coefficient scaling (LS) | UNet and UViT · CIFAR10, ImageNet 64, MS-COCO, CelebA | FID: 3.17 -> 3.07; 3.77 -> 3.39; 7.32 -> 6.89; 2.87 -> 2.73 | Improved |
| Use CS or LS instead of uniform 1/√2 scaling | UNet and UViT · CIFAR10, ImageNet 64, MS-COCO | FID: 3.14 -> 3.05; 3.66 -> 3.39; 7.19 -> 6.89 | Improved |
| Use uniform 1/√2 scaling | UViT-S/2 · CelebA | FID: 2.87 -> 3.15 | Hurt |
| Use long skip coefficient scaling rather than block-output scaling | UViT · CIFAR10 | FID: 3.46 -> 2.98 | Improved |
| Increase LS SE compression ratio from 4 or 8 to 16 (ablation) | UViT-S/2 + LS · CIFAR10 | FID: 3.06 -> 3.01; 3.10 -> 3.01 | Improved |
| Replace ReLU with ELU or Mish in the LS module (ablation) | UViT-S/2 + LS · CIFAR10 | FID: 3.01 -> 3.08 | Hurt |
| Use individual learnable coefficients instead of an LS calibration module (ablation) | UViT-S/2 + LS · CIFAR10 | FID: 3.01 -> 3.46 | Hurt |
| Replace the SE calibration module with IE (ablation) | UViT-S/2 + LS · CIFAR10 | FID: 3.01 -> 3.09 | Hurt |
| Reverse the direction of exponential CS coefficient decay (ablation) | UViT · CIFAR10, batch size 64 | FID-10k at 45k training steps: 6.65 -> 12.26 | Hurt |
| Apply CS or LS to the long skip connections | Diffusion models with UNet or UViT backbones · Multiple datasets and training settings | training speed: at least 1.5× | Improved |
Closest papers by meaning
- Analyzing and Improving the Training Dynamics of Diffusion Models (CVPR 2024 · similarity 0.90)
- Elucidating the Exposure Bias in Diffusion Models (ICLR 2024 · similarity 0.88)
- Why DiT Models Underperform as Representation Learners without Long Skip Connections (NeurIPS 2026 · similarity 0.87)
- Early-Bird Diffusion: Investigating and Leveraging Timestep-Aware Early-Bird Tickets in Diffusion Models for Efficient Training (CVPR 2025 · similarity 0.87)
- A Generalized Neural Diffusion Framework on Graphs (AAAI 2024 · similarity 0.87)
- Towards a Mechanistic Explanation of Diffusion Model Generalization (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
