Research map / Video / Video generation
PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PhyMotion, a physics-grounded 3D human motion reward (SMPL/GVHMR recovery, MuJoCo kinematic, contact/balance, and dynamic feasibility scores) that correlates better with human judgments and improves human-motion video generation when used for RL post-training of Causal Forcing and FastWan.
Problem
Existing video rewards rely on 2D perceptual signals and fail to capture 3D body state, contact, and dynamics, so they score physically implausible human motion highly and provide weak RL signal for human video generation.
Approach
Lift generated videos to 3D via SMPL-X recovery (GVHMR), retarget to a MuJoCo humanoid, and compute three interpretable feasibility scores (kinematic, contact/balance, dynamic) that are averaged into a reward for DiffusionNFT-style forward-process RL with LoRA.
Key results
- PhyMotion overall Spearman correlation with human judgments of +0.376 vs +0.262 for best VBench metric (Human Anatomy) on 1,200 pairs.
- RL post-training of Causal Forcing 1.3B with PhyMotion raises overall PhyMotion score from 0.871 to 0.902 and human preference Elo to 1621, above Wan2.2 14B (1604).
- Training overhead of 7% effective, versus 35% for HPSv3, because reward computation is pipelined with sampling.
Surprising findings
- VBench Dynamic Degree decreased substantially after PhyMotion training (-34.9% and -54.7%), which the authors attribute to large motion amplitude being rewarded as jitter.
- Dynamic Degree and Human Anatomy from VBench show near-zero or negative correlation with human judgments (Dynamic Degree negative).
Limitations
- Relies on pose recovery (GVHMR) and MuJoCo retargeting; errors in reconstruction propagate into the reward.
- Evaluated on only two 1.3B distilled backbones and a single 330-step training run for the main results.
- Feasibility thresholds (e.g., torque limits, 70 kg body mass, inertia 1.0) are hand-chosen heuristics.
Experimental setup
Datasets: Motion-X VBench prompts VBench-2.0 prompts
Benchmarks: VBench VBench-2.0 VideoAlign VideoPhy Motion-X prompts human preference study
Metrics: Spearman correlation with human judgments pairwise agreement Elo rating VBench scores VideoAlign MQ/VQ VideoPhy PC PhyMotion feasibility scores
Baselines: VBench VBench-2.0 VideoAlign VideoPhy VideoAlign-MQ HPSv3 EchoMotion
Builds on: DiffusionNFT Astrolabe GVHMR MuJoCo Causal Forcing FastWan
Closest papers by meaning
- PhysCtrl: Generative Physics for Controllable and Physics-Grounded Video Generation (NeurIPS 2025 · similarity 0.88)
- RoboScape: Physics-informed Embodied World Model (NeurIPS 2025 · similarity 0.87)
- PhysPT: Physics-aware Pretrained Transformer for Estimating Human Dynamics from Monocular Videos (CVPR 2024 · similarity 0.86)
- POMP: Physics-consistent Motion Generative Model through Phase Manifolds (CVPR 2025 · similarity 0.86)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.86)
- HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.