Research map / Video / Video generation
FIFO-Diffusion: Generating Infinite Videos from Text without Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Training-free FIFO queue-based diagonal denoising with latent partitioning and lookahead denoising lets short-clip text-to-video diffusion models generate arbitrarily long videos.
Problem
Long text-to-video generation with diffusion models trained on short clips suffers from temporal inconsistency and discontinuities in chunked autoregressive methods, and existing training-free methods have memory costs that grow with video length.
Approach
Diagonal denoising over a FIFO queue of frames with monotonically increasing noise levels, dequeuing a clean frame and enqueuing fresh noise each step; latent partitioning narrows the noise gap per block to reduce train-inference mismatch, and lookahead denoising lets noisier frames reference cleaner ones.
Key results
- FIFO-Diffusion with Latte (n=4) reaches FVD128 596.64 on UCF-101, beating PVDM-L (400-400s) at 648.4 and StyleGAN-V at 1773.4.
- Memory stays constant (about 11.2 GB on 1 GPU) from 128 to 512 frames, while FreeNoise runs out of memory at 512 frames.
- Relative MSE with latent partitioning (n=4) and lookahead denoising drops to 0.98, below the 1.0 reference of the original denoising.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FIFO-Diffusion with latent partitioning (n=4) and lookahead denoising instead of PVDM-L (400-400s) | UCF-101 video generation · UCF-101 | FVD_128: 648.4 -> 596.64 | Helped |
| Use FIFO-Diffusion with latent partitioning (n=4) and lookahead denoising instead of PVDM-L (400-400s) | UCF-101 video generation · UCF-101 | IS: 74.40{$\pm$1.25} -> 74.44{$\pm$1.17} | No clear effect |
| Use FIFO-Diffusion instead of FreeNoise | long video generation · 128 frames | memory usage [MB]: 26,163 -> 11,245 | Helped |
| Use FIFO-Diffusion instead of FreeNoise | long video generation · 256 frames | memory usage [MB]: 44,683 -> 11,245 | Helped |
| Increase target video length with FIFO-Diffusion | FIFO-Diffusion (1 GPU) · 128 to 512 frames | memory usage [MB]: 11,245 -> 11,245 | No clear effect |
| Use FIFO-Diffusion instead of Gen-L-Video | long video generation · long video generation | inference time [s/frame]: 22.07 -> 12.37 | Helped |
| Parallelize FIFO-Diffusion across 8 GPUs instead of using 1 GPU | FIFO-Diffusion · long video generation | inference time [s/frame]: 12.37 -> 1.84 | Helped |
| Add lookahead denoising without latent partitioning (ablation) | diagonal denoising · VideoCrafter1 noise prediction | relative MSE loss: 1.09 -> 1.01 | Helped |
| Add latent partitioning with 2 partitions (ablation) | diagonal denoising · VideoCrafter1 noise prediction | relative MSE loss: 1.09 -> 1.04 | Helped |
| Add latent partitioning with 4 partitions (ablation) | diagonal denoising · VideoCrafter1 noise prediction | relative MSE loss: 1.09 -> 1.02 | Helped |
| Add lookahead denoising with latent partitioning (2 partitions) (ablation) | diagonal denoising with latent partitioning · VideoCrafter1 noise prediction | relative MSE loss: 1.04 -> 0.99 | Helped |
| Add lookahead denoising with latent partitioning (4 partitions) (ablation) | diagonal denoising with latent partitioning · VideoCrafter1 noise prediction | relative MSE loss: 1.02 -> 0.98 | Helped |
Surprising findings
- Training-free FIFO-Diffusion outperforms the trained chunked autoregressive PVDM-L on FVD128 on UCF-101.
Limitations
- Training-inference gap persists after latent partitioning because the model's input distribution still differs from training.
- Lookahead denoising doubles per-step computation and latent partitioning increases the number of inference steps.
- Evaluation on the unconditional UCF-101 benchmark uses Latte and relies on user studies for text-conditional quality.
Experimental setup
Datasets: UCF-101 MSR-VTT
Benchmarks: UCF-101
Metrics: FVD128 IS relative MSE memory usage inference time per frame optical flow magnitude
Baselines: FreeNoise Gen-L-Video LaVie + SEINE PVDM-L (400-400s) StyleGAN-V VIDM
Builds on: DDIM classifier-free guidance diffusion Video Diffusion Models
Closest papers by meaning
- Ouroboros-Diffusion: Exploring Consistent Content Generation in Tuning-free Long Video Diffusion (AAAI 2025 · similarity 0.91)
- FreeNoise: Tuning-Free Longer Video Diffusion via Noise Rescheduling (ICLR 2024 · similarity 0.90)
- AdaDiff: Adaptive Step Selection for Fast Diffusion Models (AAAI 2025 · similarity 0.89)
- FreeLong: Training-Free Long Video Generation with SpectralBlend Temporal Attention (NeurIPS 2024 · similarity 0.89)
- AR-Diffusion: Asynchronous Video Generation with Auto-Regressive Diffusion (CVPR 2025 · similarity 0.88)
- Equilibrium Forcing: Adaptive Video Generation Without Noise Conditioning (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.