Research map / Video / Video generation
$h$-control: Training-Free Camera Control via Block-Conditional Gibbs Refinement
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes h-control, a training-free camera-control sampler for pretrained flow-matching video models that wraps hard masked latent replacement with an inner block-conditional pseudo-Gibbs refinement over 3D patches with a Welford-based freeze gate.
Problem
Training-free camera control for video flow-matching models trades off trajectory adherence against visual quality; hard replacement gives tight control but artifacts, while soft guidance needs heuristic strength tuning.
Approach
Augment each outer hard-replacement step with an inner pseudo-Gibbs (generalized DAE) chain on the unobserved latent complement at the same noise level, partitioned into 3D patches that are frozen adaptively once their running std stabilizes, claimed to converge to the partial-observation conditional law.
Key results
- On RealEstate10K, h-control reaches FVD 129.25 vs 157.08 for the best training-free baseline (Coarse-Guided), a 17.7% reduction.
- On DAVIS, h-control reaches FVD 356.73, beating the best training-based TrajectoryCrafter (576.30) and WorldForge (383.53) among training-free methods.
- Ablation on DAVIS: pseudo-Gibbs refinement alone lowers FVD from 392.13 to 382.26; full sampler reaches 356.73.
Surprising findings
- The patch-freeze gate leaves FVD unchanged on its own (382.26) while improving RRE, which the authors attribute to protecting already-mixed structure.
Limitations
- Locality/order-2 Markov dependence calibrated to Wan 2.2 latent and must be re-measured per backbone.
- Outer step inherits depth-and-reproject warping errors; severe depth errors degrade conditioning.
- Gibbs refinement requires full forward passes over all tokens, so no wall-clock acceleration from freezing.
Experimental setup
Datasets: RealEstate10K DAVIS
Benchmarks: RealEstate10K DAVIS
Metrics: FVD ATE RRE RTE LPIPS SSIM CLIP-f CLIP-v
Baselines: WorldForge TTM RePaint Coarse-Guided TrajectoryAttention TrajectoryCrafter ReCamMaster
Builds on: Bengio generalized denoising auto-encoder DPS Doob h-transform SRVS TTM
Closest papers by meaning
- Training-free Camera Control for Video Generation (ICLR 2025 · similarity 0.87)
- Understanding and Improving Training-free Loss-based Diffusion Guidance (NeurIPS 2024 · similarity 0.87)
- AC3D: Analyzing and Improving 3D Camera Control in Video Diffusion Transformers (CVPR 2025 · similarity 0.87)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.86)
- On the Guidance of Flow Matching (ICML 2025 · similarity 0.86)
- History-Guided Video Diffusion (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.