Research map / Video / Video generation
Frame Guidance: Training-Free Guidance for Frame-Level Control in Video Diffusion Model
Sangwon Jang, Taekyung Ki, Jaehyeong Jo, Jaehong Yoon, Soo Ye Kim, Zhe Lin, Sung Ju Hwang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use video latent optimization (deterministic early, stochastic later) instead of time-travel only (ablation) | Frame Guidance · keyframe-guided generation | FVD: 778.4 -> 577.1 FVD; 57.37 -> 55.60 FID | Improved |
| add stochastic later-stage updates to deterministic-only updates (full VLO) (ablation) | Frame Guidance · keyframe-guided generation | FVD: 637.3 -> 577.1 FVD; 56.61 -> 55.60 FID | Improved |
| decode only a 3-latent slice for the guidance loss (latent slicing) (ablation) | Frame Guidance on CausalVAE · GPU memory | memory: up to 15x reduction | Improved |
| add 2x spatial down-sampling of latents to latent slicing (ablation) | Frame Guidance · GPU memory | memory: up to 60x reduction | Improved |
| backpropagate guidance loss through the denoising network instead of shortcut update (ablation) | Frame Guidance · keyframe-guided generation | temporal coherence (qualitative) | Improved |
| use a 2-latent slice instead of 3-latent slice for reconstruction (ablation) | latent slicing · video reconstruction | reconstruction quality (qualitative) | Hurt |
| vary t_E (end of deterministic stage) (ablation) | VLO · DAVIS (20 videos) keyframe generation | FID/FVD: best at t_E=6 | No clear effect |
| apply Frame Guidance to pre-trained I2V models for keyframe guidance | CogX and Wan · DAVIS and Pexels keyframe generation | FID/FVD | Improved |
| apply Frame Guidance on top of CogX-Interp | CogX-Interp · keyframe-guided generation | FID/FVD | Improved |
| apply Frame Guidance to Wan I2V | Wan-14B I2V · keyframe-guided generation | FID/FVD | Improved |
| guide Sobel edge maps instead of RGB keyframes | Frame Guidance · edge-map guidance | control quality (qualitative) | Hurt |
Closest papers by meaning
- Training-free Camera Control for Video Generation (ICLR 2025 · similarity 0.89)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.89)
- Generative Inbetweening: Adapting Image-to-Video Models for Keyframe Interpolation (ICLR 2025 · similarity 0.89)
- Ctrl-Adapter: An Efficient and Versatile Framework for Adapting Diverse Controls to Any Diffusion Model (ICLR 2025 · similarity 0.89)
- ControlVideo: Training-free Controllable Text-to-video Generation (ICLR 2024 · similarity 0.89)
- BIVDiff: A Training-Free Framework for General-Purpose Video Synthesis via Bridging Image and Video Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
