AI Paper Atlas

Research map / Video / Video generation

Video-As-Prompt: Unified Semantic Control for Video Generation

Yuxuan Bian, Xin Chen, Zenan Li, Tiancheng Zhi, Shen Sang, Linjie Luo, Qiang Xu

ICLR 2026 · Video › Video generation · approach: Trajectory-guided motion control

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use mixture-of-transformers (frozen DiT + parallel expert with full attention) instead of single-branch full finetuning (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 68.74 -> 70.44Improved
use MoT instead of single-branch LoRA finetuning (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testDynamic degree: 72.92 -> 77.08 dynamic degree; alignment 69.08 -> 70.44Mixed
use bidirectional full attention instead of unidirectional cross-attention injection (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 67.16 -> 70.44Improved
use MoT full attention instead of residual addition injection (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 55.99 -> 70.44Improved
add temporally biased RoPE instead of identical RoPE for reference and target (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 68.98 -> 70.44Improved
add a width bias (reference placed left) in addition to temporal bias (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 70.44 -> 69.05Hurt
scale training pairs from 1K to 100K (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 63.91 (1K) -> 66.28 (10K) -> 68.23 (50K) -> 70.44 (100K)Improved
swap backbone to Wan2.1-I2V-14B with expert in 1/4 of layers (ablation)VAP · VAP-Data testDynamic degree: dynamic degree 77.08 -> 79.17; alignment 70.44 -> 70.23Mixed
place MoT expert in all layers instead of only odd layers (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 70.22 -> 70.44Improved
add noise to the reference video prompt (ablation)VAP on CogVideoX-I2V-5B · VAP-Data testSemantic alignment score: 70.44 -> 70.18Hurt
use more reference videos (1-3) as promptsVAP · multi-reference testqualitativeNo clear effect
use structure-controlled VACE with raw video condition instead of depth/optical flowVACE · VAP-Data testSemantic alignment score: 46.71 (flow) -> 43.35 (depth) -> 35.38 (original)Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy