Research map / Video / Video generation
UFO: Enhancing Diffusion-Based Video Generation with a Uniform Frame Organizer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UFO, a non-invasive plug-in of lightweight adapters with adjustable intensity that improves consistency and image quality of diffusion-based text-to-video models without changing base weights.
Problem
Diffusion-based video generation models suffer from weak subject/background consistency, temporal flickering, blurring, and declining quality over longer videos.
Approach
Attach low-rank-style detection/correction adapters with a tunable global intensity to frozen DiT layers; train them to make frames static from image-text pairs, then use a low intensity at inference to keep motion while gaining consistency. Adapters can be transferred across same-spec models and combined with style adapters.
Key results
- On OpenSora-V1.2 at 240x426, UFO with alpha=0.2 raised VBench temporal quality from 95.55% to 97.01% and subject consistency from 93.00% to 95.08%.
- On EasyAnimate-V2 at 576x1008, alpha=0.15 raised TQ from 96.41% to 98.57% and frame-wise quality from 63.57% to 64.98%.
- Adapter dimension d=4 gave best trade-off; UFO adds about 0.3-0.5% of trainable parameters (about 0.005x per the text) and 0.24-0.58% inference time overhead.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add consistency UFO at α=0.2 | OpenSora-V1.2 · VBench, Open 240 × 426 | TQ: 95.55% -> 97.01% | Helped |
| Add consistency UFO at α=0.15 | EasyAnimate-V2 · VBench, Easy 384 × 672 | TQ: 97.07% -> 99.02% | Helped |
| Increase adapter dimension from d=1 to d=4 (ablation) | Consistency UFO on OpenSora-V1.2 · VBench, Open 480 × 854 | TQ: 95.37% -> 96.35% | Helped |
| Increase adapter dimension from d=4 to d=8 (ablation) | Consistency UFO on OpenSora-V1.2 · VBench, Open 480 × 854 | TQ: 96.35% -> 96.29% | Hurt |
| Increase adapter dimension from d=4 to d=64 (ablation) | Consistency UFO on OpenSora-V1.2 · VBench, Open 480 × 854 | TQ: 96.35% -> 96.03% | Hurt |
| Increase adapter dimension from d=1 to d=4 (ablation) | Consistency UFO on EasyAnimate-V2 · VBench, Easy 576 × 1008 | TQ: 96.53% -> 97.53% | Helped |
| Increase adapter dimension from d=4 to d=8 (ablation) | Consistency UFO on EasyAnimate-V2 · VBench, Easy 576 × 1008 | TQ: 97.53% -> 97.49% | No clear effect |
| Increase adapter dimension from d=4 to d=64 (ablation) | Consistency UFO on EasyAnimate-V2 · VBench, Easy 576 × 1008 | TQ: 97.53% -> 97.14% | Hurt |
| Transfer a UFO trained on another same-specification model instead of retraining (ablation) | EasyAnimate-V2 · VBench, 384 × 672, α=0.07 | TQ: 98.20% vs 98.18% | No clear effect |
| Increase consistency UFO intensity from α=0 to α=0.1 (ablation) | OpenSora-V1.2 · VBench, 480 × 854, 8-second videos | TQ: +1.25% | Helped |
| Increase consistency UFO intensity from α=0 to α=0.07 (ablation) | EasyAnimate-V2 · VBench, 384 × 672, 6-second videos | TQ: +1.47% | Helped |
| Use aggressive consistency UFO intensity (ablation) | Consistency UFO · Video generation | motion | Hurt |
Surprising findings
- A consistency UFO trained on one model transfers to another same-spec model with performance almost identical to retraining.
- Higher intensity alpha makes many videos nearly static, so the gains in consistency come at the cost of motion.
Limitations
- Output is bounded by the base model; cannot fix scene transitions or very poor image quality.
- Higher intensity reduces motion, causing many videos to become nearly static.
- Evaluation restricted to two open-source models and VBench metrics; semantic consistency not targeted.
Experimental setup
Datasets: LAION-Aesthetics V2 internet stylized videos (Pixel Art, oil painting, animated, black and white) VBench prompts
Benchmarks: VBench
Metrics: subject consistency background consistency temporal flickering motion smoothness aesthetic quality imaging quality semantic quality excluded count (optical flow based)
Baselines: OpenSora-V1.2 EasyAnimate-V2 Transferred consistency UFO Retrained consistency UFO
Builds on: LoRA latent diffusion models VBench
Closest papers by meaning
- IllumiCraft: Unified Geometry and Illumination Diffusion for Controllable Video Generation (NeurIPS 2025 · similarity 0.85)
- Generative Rendering: Controllable 4D-Guided Video Generation with 2D Diffusion Models (CVPR 2024 · similarity 0.83)
- TweedieMix: Improving Multi-Concept Fusion for Diffusion-based Image/Video Generation (ICLR 2025 · similarity 0.83)
- Track4Gen: Teaching Video Diffusion Models to Track Points Improves Video Generation (CVPR 2025 · similarity 0.83)
- Collaborative Video Diffusion: Consistent Multi-video Generation with Camera Control (NeurIPS 2024 · similarity 0.83)
- EIDT-V: Exploiting Intersections in Diffusion Trajectories for Model-Agnostic, Zero-Shot, Training-Free Text-to-Video Generation (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.