Research map / Video / Video generation
Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DyMoS, a training-free, model-agnostic method that subtracts a scalar bias from attention logits of non-reference query tokens to reference-frame keys during early denoising, to reduce static motion in image-to-video generation.
Problem
Image-to-video diffusion models often generate overly static videos compared to text-to-video models, and prior fixes either need retraining or degrade fidelity to the reference image.
Approach
Identifies "reference-frame dominance" (excess self-attention from generated frames to reference-frame keys, especially in early denoising steps) as a cause of motion suppression, and rebalances it by adding a logit bias only during the first fraction of sampling steps, with one scalar gamma controlling motion strength.
Key results
- On Wan 2.2-14B on VBench-I2V, DyMoS raises VBench Dynamic Degree from 51.7 (baseline) and 55.3 (ALG) to 64.8 while keeping Video Quality at 57.7 vs 58.0 baseline.
- On CogVideoX-5B, DyMoS raises VBench Dynamic Degree from 30.9 to 52.9 with ViCLIP 0.2652 vs 0.2612 baseline.
- Inference overhead is 1.2% to 4.3% across backbones (e.g., +1.2% on Wan 2.2), versus up to +10.3% for ALG.
Surprising findings
- Video Quality stays nearly constant across a broad range of modulation strengths while Dynamic Degree rises substantially.
- Negative gamma (strengthening reference dominance) makes videos more static, confirming the causal role of reference-frame attention.
Limitations
- Overly large modulation strength reduces appearance consistency and temporal stability.
- Hyperparameters (gamma, lambda) are fixed per backbone and not adapted automatically to prompts or motion types.
Experimental setup
Datasets: VBench-I2V PE Video Dataset (PVD) VidProM T2V-CompBench
Benchmarks: VBench-I2V PVD VidProM
Metrics: Dynamic Degree (VBench) Dynamic Degree (VideoScore) Video Quality ViCLIP similarity VisionReward user study preference
Baselines: Vanilla I2V backbone ALG
Builds on: classifier-free guidance ALG VBench flow matching
Closest papers by meaning
- Extrapolating and Decoupling Image-to-Video Generation Models: Motion Modeling is Easier Than You Think (CVPR 2025 · similarity 0.88)
- Identifying and Solving Conditional Image Leakage in Image-to-Video Diffusion Model (NeurIPS 2024 · similarity 0.87)
- FlowMo: Variance-Based Flow Guidance for Coherent Motion in Video Generation (NeurIPS 2025 · similarity 0.87)
- MotionPro: A Precise Motion Controller for Image-to-Video Generation (CVPR 2025 · similarity 0.87)
- DynaTokens: Teaching Dynamics to Camera-Controlled Video Models at Test Time (NeurIPS 2026 · similarity 0.86)
- VideoJAM: Joint Appearance-Motion Representations for Enhanced Motion Generation in Video Models (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.