Research map / 3D vision / 3D generation

ONE-SHOT: Compositional Human-Environment Video Synthesis via Spatial-Decoupled Motion Injection and Hybrid Context Integration

NeurIPS 2026 · 3D vision › 3D generation · approach: Scene-aware human motion synthesis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ONE-SHOT, a LoRA-based ControlNet-style framework on frozen Wan2.1-14B that composes human motion (canonical-space SMPL-X cross-attention with Dynamic-Grounded-RoPE), scene point-cloud RGB-D projections, and static/dynamic hybrid context for minute-level human-environment video synthesis.

Problem

Independent control of human subjects and scenes in video generation, avoiding heavy explicit 3D human-scene alignment, preserving generative flexibility, and maintaining consistency over long horizons.

Approach

Decouple human motion from environment by injecting canonical-space SMPL-X motion tokens via a dedicated cross-attention layer, align the two spatial domains with scale-grounded RoPE using a coarse 2D bbox instead of 3D alignment, and maintain consistency with static identity references plus dynamic context memory.

Key results

• On Traj100 self-reconstruction, ONE-SHOT reaches FID 16.88 and FVD 181.17, versus 18.89 / 193.89 for RealisMotion (best baseline).
• On the cross-composition test set, ONE-SHOT achieves FID 150.98 and FVD 1382.4, best among compared methods, with Motion Smoothness 0.993.
• Removing Dynamic-Grounded-RoPE scale grounding raises FID from 16.88 to 22.34 and FVD from 181.17 to 229.76 on Traj100.

Surprising findings

• RealisMotion attains higher subject similarity (0.356) than ONE-SHOT (0.317) on Traj100, which the authors acknowledge.

Limitations

• Quality depends on reconstructed scene point clouds and camera trajectories; noisy reconstruction or human removal degrades results.
• Errors from the human estimation module (Human3R) propagate to outputs, and extreme bbox misplacement can cause failures.
• Some temporal drift still accumulates in very long videos.

Experimental setup

Datasets: EMDB2 MotionX ARKitScenes self-collected web videos

Benchmarks: Traj100 self-collected cross-composition test set

Metrics: FID FVD Motion Smoothness Background Consistency Subject Similarity Face Similarity

Baselines: WanAnimate RealisDance Uni3C RealisMotion WAN-VACE (fine-tuned)

Builds on: Wan2.1 ControlNet LoRA RealisMotion

Closest papers by meaning

• TokenMotion: Decoupled Motion Control via Token Disentanglement for Human-centric Video Generation (CVPR 2025 · similarity 0.88)
• Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation (NeurIPS 2026 · similarity 0.88)
• Wan-Move: Motion-controllable Video Generation via Latent Trajectory Guidance (NeurIPS 2025 · similarity 0.87)
• IM-Zero: Instance-level Motion Controllable Video Generation in a Zero-shot Manner (CVPR 2025 · similarity 0.87)
• HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.87)
• Frame In-N-Out: Unbounded Controllable Image-to-Video Generation (NeurIPS 2025 · similarity 0.87)