Research map / Video / Video generation
OptiWorld: Optimal Control for Video World Generation under Physical Constraints
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OptiWorld, which adds an inference-time optimal-control planning layer (geometry on a constraint-induced Riemannian manifold) between zero-shot 3D scene understanding and 3D-trajectory-conditioned video rendering, improving safety, smoothness, and efficiency of generated motion.
Problem
Video generation models produce plausible but possibly unsafe, jerky, inefficient or physically inconsistent object motion because they do not optimize dynamics under physical constraints before rendering.
Approach
Plan an optimal 3D object trajectory before rendering by encoding goal reaching, safety (geometric and VLM-assigned semantic hazards), efficiency and smoothness into a continuous Riemannian metric and potential, solve it with voxel A* seed plus gradient refinement, and feed the trajectory as tracking control to a 3D-aware video renderer (Diffusion as Shader).
Key results
- On goal-conditioned I2V (OptiBench-I2V, 60 scenes), OptiWorld reaches goal error 0.310 and violation-among-success 0.260, versus 0.331 / 0.286 for the best baselines (VLIPP, Cosmos-Predict2.5).
- On video dynamics editing (OptiBench-V2V, 30 videos), OptiWorld lowers track deviation to 0.113 (vs 0.123 for Wan2.1 FLF2V) and energy to 0.0236 (vs 0.0269 source video).
- Ablation: removing the safety term raises violation-among-success from 0.304 to 0.589 in planning diagnostics; removing smoothness raises acceleration from 2.40 to 3.20 (x10^-4).
Surprising findings
- Planning-stage ablation shows removing the manifold (flat Euclidean objective) raises violation rate, indicating the geometric formulation matters beyond the individual penalties.
Limitations
- Final video quality is bounded by the DaS renderer, which can produce artifacts and deformation for large motions or thin/reflective objects.
- Pipeline is offline and not real-time.
- Evaluation uses small self-collected benchmarks (60 and 30 clips) with a metric set defined by the authors.
Experimental setup
Datasets: DROID
Benchmarks: OptiBench-I2V OptiBench-V2V VBench
Metrics: goal error violation among successful cases acceleration jerk energy path length track deviation VBench motion smoothness VBench background consistency VBench temporal flickering
Baselines: HunyuanVideo-1.5 Wan2.2 Cosmos-Predict2.5 VLIPP Wan2.1 FLF2V Go-with-the-Flow
Builds on: Diffusion as Shader VLIPP MoGe2 SAM3 A*
Closest papers by meaning
- DriveDreamer-Policy: A Geometry-Grounded World-Action Model for Unified Generation and Planning (NeurIPS 2026 · similarity 0.87)
- Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
- Motion Forcing: Decoupling Ego and Object Motion via Sparse Inputs for Structured Video Generation (NeurIPS 2026 · similarity 0.87)
- FLIP: Flow-Centric Generative Planning as General-Purpose Manipulation World Model (ICLR 2025 · similarity 0.86)
- GEN3C: 3D-Informed World-Consistent Video Generation with Precise Camera Control (CVPR 2025 · similarity 0.86)
- EA-WM: Event-Aware Generative World Model with Structured Kinematic-to-Visual Action Fields (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.