Research map / Architectures and efficiency / RNNs and normalization
Towards Precise Scaling Laws for Video Diffusion Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives scaling laws for video diffusion transformers that predict optimal batch size and learning rate as functions of model size and training tokens, then uses optimal hyperparameters to fit more precise model-size and loss scaling laws.
Problem
Video diffusion transformers are very costly to train, and existing LLM-style scaling laws ignore hyperparameter choice (batch size, learning rate), which makes predictions of optimal model size and loss imprecise for video generation.
Approach
Model optimal batch size and learning rate as power laws in training tokens and model size (derived from SGD convergence analysis), then fit model-size/compute and loss scaling laws only under these optimal hyperparameters, showing fixed suboptimal hyperparameters distort the fits.
Key results
- Optimal-hyperparameter scaling law predicts an optimal model 39.9% smaller (40.1% lower inference cost) than fixed-hyperparameter predictions at 1e10 TFlops budget with comparable loss.
- Fitting MSE drops from 4.31e-7 with fixed suboptimal hyperparameters to 2.35e-7 with optimal hyperparameters (45.5% reduction).
- Slope deviation of predicted optimal model size vs. empirical drops from 30.26% (fixed) to 3.57% (optimal hyperparameters).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the optimal-hyperparameter scaling law instead of fixed suboptimal hyperparameters (ablation) | Video DiT scaling law · Validation set | mean squared error: 4.31 ⇥ 10−7 -> 2.35 ⇥ 10−7 | Helped |
| Use optimal hyperparameters to predict optimal model size (ablation) | Video DiT scaling law · Optimal model size vs compute budget | slope deviation: 30.26% -> 3.57% | Helped |
| Use the optimal-hyperparameter scaling law rather than fixed suboptimal hyperparameters (ablation) | Video DiT at a compute budget of 1 × 10¹⁰ TFlops · Validation set | inference cost: 40.1% | Helped |
| Use optimal hyperparameters rather than fixed suboptimal hyperparameters | Video DiT · Validation set | validation loss: 0.5403 vs. 0.5071 | Helped |
| Use optimal hyperparameters rather than fixed suboptimal hyperparameters | Video DiT · Validation set | validation loss: 0.03% and 0.15% | Helped |
Surprising findings
- Video diffusion models are more sensitive to learning rate and batch size than language models, so fixed-hyperparameter scaling laws give inaccurate optimal model size predictions.
Limitations
- Only constant learning rate schedule was studied; learning rate decay not explored.
- Evaluated only with validation loss due to lack of standard video quality metric; no external benchmarks.
- Small, low-resolution models; results may not transfer to higher resolution or larger models; resolution and frame rate effects not studied.
Experimental setup
Datasets: private video dataset (partitioned into train and validation)
Benchmarks: validation loss
Metrics: validation loss MSE of fit predicted optimal model size error VBench aesthetic score
Baselines: Chinchilla scaling law DeepSeek scaling law OpenAI scaling law fixed suboptimal hyperparameter scaling
Builds on: Chinchilla scaling law DeepSeek scaling law OpenAI scaling law
Closest papers by meaning
- Scaling Diffusion Transformers Efficiently via $\mu$P (NeurIPS 2025 · similarity 0.92)
- OSV: One Step is Enough for High-Quality Image to Video Generation (CVPR 2025 · similarity 0.90)
- Scaling Properties of Diffusion Models For Perceptual Tasks (CVPR 2025 · similarity 0.90)
- From Slow Bidirectional to Fast Autoregressive Video Diffusion Models (CVPR 2025 · similarity 0.90)
- ViDiT-Q: Efficient and Accurate Quantization of Diffusion Transformers for Image and Video Generation (ICLR 2025 · similarity 0.90)
- IV-mixed Sampler: Leveraging Image Diffusion Models for Enhanced Video Synthesis (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.