AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

ScaleLong: Towards More Stable Training of Diffusion Model via Scaling Network Long Skip Connection

Zhongzhan Huang, Pan Zhou, Shuicheng YAN, Liang Lin

NeurIPS 2023 · Architectures and efficiency › RNNs and normalization · approach: Theory of depth and width · 5 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Add exponentially decaying long skip coefficient scaling (CS)UNet and UViT · CIFAR10, ImageNet 64, MS-COCO, CelebAFID: 3.17 -> 3.05; 3.77 -> 3.48; 7.32 -> 7.04; 2.87 -> 2.78Improved
Add learnable long skip coefficient scaling (LS)UNet and UViT · CIFAR10, ImageNet 64, MS-COCO, CelebAFID: 3.17 -> 3.07; 3.77 -> 3.39; 7.32 -> 6.89; 2.87 -> 2.73Improved
Use CS or LS instead of uniform 1/√2 scalingUNet and UViT · CIFAR10, ImageNet 64, MS-COCOFID: 3.14 -> 3.05; 3.66 -> 3.39; 7.19 -> 6.89Improved
Use uniform 1/√2 scalingUViT-S/2 · CelebAFID: 2.87 -> 3.15Hurt
Use long skip coefficient scaling rather than block-output scalingUViT · CIFAR10FID: 3.46 -> 2.98Improved
Increase LS SE compression ratio from 4 or 8 to 16 (ablation)UViT-S/2 + LS · CIFAR10FID: 3.06 -> 3.01; 3.10 -> 3.01Improved
Replace ReLU with ELU or Mish in the LS module (ablation)UViT-S/2 + LS · CIFAR10FID: 3.01 -> 3.08Hurt
Use individual learnable coefficients instead of an LS calibration module (ablation)UViT-S/2 + LS · CIFAR10FID: 3.01 -> 3.46Hurt
Replace the SE calibration module with IE (ablation)UViT-S/2 + LS · CIFAR10FID: 3.01 -> 3.09Hurt
Reverse the direction of exponential CS coefficient decay (ablation)UViT · CIFAR10, batch size 64FID-10k at 45k training steps: 6.65 -> 12.26Hurt
Apply CS or LS to the long skip connectionsDiffusion models with UNet or UViT backbones · Multiple datasets and training settingstraining speed: at least 1.5×Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy