Research map / Video / Video generation
FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeNoise, a tuning-free method that reschedules initial noise with local shuffling and fuses overlapping temporal-attention windows to extend pretrained video diffusion models to longer videos, plus a motion injection scheme for multi-prompt generation.
Problem
Pretrained text-to-video diffusion models trained on a fixed small number of frames degrade when generating longer videos, and they support only single text conditions even though real videos often need multiple prompts over time.
Approach
Construct long initial noise by locally shuffling a fixed set of training-length noise frames (giving long-range correlation with i.i.d.-like windows), apply temporal attention only within sliding windows with distance-weighted fusion, and inject a second prompt only during shape-related denoising steps and in decoder cross-attention layers.
Key results
- On 64-frame generation, FreeNoise achieves FVD 85.83 and KVD 7.06 versus 177.63 / 21.06 for Gen-L-Video, with CLIP-SIM 0.9732 vs 0.9370.
- FreeNoise adds roughly 17% inference time (25.75s vs 21.97s direct), while Gen-L-Video costs 77.89s.
- In a 27-user study, FreeNoise is preferred by 57.72% for content consistency, 55.56% for video quality, and 51.23% for video-text alignment.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use noise rescheduling with local noise shuffling and window-based attention fusion | FreeNoise on VideoCrafter · Longer video generation (64 frames) | FVD: 177.63 -> 85.83 | Helped |
| Use noise rescheduling with local noise shuffling and window-based attention fusion | FreeNoise on VideoCrafter · Longer video generation (64 frames) | KVD: 21.06 -> 7.06 | Helped |
| Use noise rescheduling with local noise shuffling and window-based attention fusion | FreeNoise on VideoCrafter · Longer video generation (64 frames) | CLIP-SIM: 0.9370 -> 0.9732 | Helped |
| Use noise rescheduling with local noise shuffling and window-based attention fusion | FreeNoise on VideoCrafter · Longer video generation (64 frames) | Inference time: 21.97s -> 25.75s | Hurt |
| Use noise rescheduling with local noise shuffling and window-based attention fusion instead of GenL | Longer video generation (64 frames) · Longer video generation (64 frames) | Inference time: 77.89s -> 25.75s | Helped |
| Use FreeNoise | VideoCrafter · User study: content consistency | Vote share: 24.38% -> 57.72% | Helped |
| Use FreeNoise | VideoCrafter · User study: video quality | Vote share: 26.85% -> 55.56% | Helped |
| Use FreeNoise | VideoCrafter · User study: video-text alignment | Vote share: 29.63% -> 51.23% | Helped |
| Remove noise rescheduling (ablation) | FreeNoise · Longer video generation | Content consistency | Hurt |
| Increase local noise shuffle unit and window stride from 4 to 8 (ablation) | FreeNoise · Longer video generation | Content consistency | Hurt |
| Allow the first prompt to control only the decoder part (ablation) | Motion Injection · Multi-prompt video generation | Motion continuity | Hurt |
| Allow the first prompt to control both encoder and decoder parts (ablation) | Motion Injection · Multi-prompt video generation | Motion continuity | Hurt |
Surprising findings
- Temporal attention is order-independent while temporal convolution is order-dependent; per-frame noises set the overall appearance while their order shapes content.
- Naive averaging of overlapping window outputs causes dramatic variation at window boundaries, motivating distance-weighted fusion.
Limitations
- Repeated locally shuffled noise weakens the introduction of new content as video length grows, limiting subject displacement.
- Performance is bounded by the base model, which handles subjects moving off or within the screen poorly.
- Evaluated only at 256x256 resolution with 16-frame training-length base models.
Experimental setup
Datasets: EvalCrafter prompts
Benchmarks: EvalCrafter
Metrics: FVD KVD CLIP-SIM inference time user study preference
Baselines: Direct sampling Sliding window Gen-L-Video
Builds on: VideoLDM Latent Diffusion Models DDIM MasaCtrl
Closest papers by meaning
- FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention (NeurIPS 2024 · similarity 0.92)
- Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (AAAI 2025 · similarity 0.90)
- FIFO-Diffusion: Generating Infinite Videos from Text without Training (NeurIPS 2024 · similarity 0.90)
- LongDiff: Training-Free Long Video Generation in One Go (CVPR 2025 · similarity 0.90)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.90)
- FreqPrior: Improving Video Diffusion Models with Frequency Filtering Gaussian Noise (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.