Research map / Video / Video generation
Aligning Human Motion Generation with Human Perceptions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds MotionPercept-style human perceptual preference dataset (52.6K annotated comparisons of generated motions) and trains a MotionCritic model on it, which is then used as a metric and as a reward-style fine-tuning signal for MDM.
Problem
Automatic metrics for human motion generation (error vs ground truth, heuristics, FID) do not align well with human perceptions of motion quality, and cannot provide instance-level supervision for generators.
Approach
Collect large-scale human multiple-choice preference annotations over groups of generated motions, train a critic on pairwise Bradley-Terry preference loss, and plug the critic into diffusion motion generator fine-tuning (ReFL-style) as a perceptual reward.
Key results
- Critic reaches 85.07% accuracy on the MDM test set and 81.43% on the FLAME test set, versus 71.78% for the best heuristic (Person-Ground Contact) and 49.40% for MoBERT.
- Fine-tuning MDM with the critic for about 700 steps raises the critic score from -1.64 to 3.45 and improves user-study Elo ratings, while FID does not track the improvement.
- Fine-tuning uses roughly 800 iterations, about 0.23% of the 350K pretraining steps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MotionCritic instead of Person-Ground Contact to rank motion quality | motion evaluation on MDM test data · MotionCritic MDM test set | accuracy: 71.78 -> 85.07 | Helped |
| Use MotionCritic instead of NDMS to estimate preference likelihood | motion evaluation on MDM test data · MotionCritic MDM test set | Log Loss: 0.6519 -> 0.5486 | Helped |
| Use MotionCritic instead of Acceleration to rank motion quality | motion evaluation on FLAME test data · MotionCritic FLAME test set | accuracy: 66.67 -> 81.43 | Helped |
| Use MotionCritic instead of Jerk to estimate preference likelihood | motion evaluation on FLAME test data · MotionCritic FLAME test set | Log Loss: 0.5984 -> 0.5758 | Helped |
| Fine-tune MDM with MotionCritic supervision for 600 steps (ablation) | MDM trained on HumanAct12 · HumanAct12 test set | MotionCritic score: -1.64 -> 2.78 | Helped |
| Fine-tune MDM with MotionCritic supervision for 600 steps (ablation) | MDM trained on HumanAct12 · HumanAct12 test set | FID: 0.13 -> 0.18 | Helped |
| Fine-tune MDM with MotionCritic supervision for 600 steps (ablation) | MDM trained on HumanAct12 · HumanAct12 test set | PFC: 0.00095 -> 0.0013 | Hurt |
| Fine-tune MDM with MotionCritic supervision for 600 steps (ablation) | MDM trained on HumanAct12 · HumanAct12 test set | Diversity: 6.66 -> 6.68 | Helped |
| Fine-tune MDM with MotionCritic supervision for 600 steps (ablation) | MDM trained on HumanAct12 · HumanAct12 test set | Multimodality: 2.15 -> 2.42 | Helped |
| Fine-tune MDM with MotionCritic supervision | MDM trained on HumanAct12 · HumanAct12 test set user study | user preference | Helped |
Surprising findings
- FID correlates poorly with human preference: on HumanAct12 GT groups, FID does not track user-study Elo ratings while the critic score does.
- Critic-based fine-tuning raised the critic score and user preference while FID worsened at some steps (e.g., FID 0.54 at step 700).
Limitations
- Critic energy landscape is non-smooth, so it is a poor distance measure and hard to optimize against; critic should not be used simultaneously as metric and loss.
- Does not explicitly model physical and biomechanical plausibility.
- Errors concentrate in cases where critic scores of the two motions are close and human annotators also disagree.
Experimental setup
Datasets: MotionCritic perceptual dataset (MDM and FLAME subsets) HumanAct12 UESTC HumanML3D AMASS
Benchmarks: MotionPercept test set (MDM and FLAME) HumanAct12 test set
Metrics: accuracy log loss FID PFC Elo rating user study win rate accuracy (action recognition) diversity multimodality
Baselines: Root AVE Root AE Joint AVE Joint AE Jerk Acceleration Person-Ground Contact Foot-Floor Penetration Physical Foot Contact PoseNDF NPSS NDMS
Builds on: MDM ReFL Bradley-Terry model DSTformer
Closest papers by meaning
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.87)
- WorldSimBench: Towards Video Generation Models as World Simulators (ICML 2025 · similarity 0.87)
- Scaling Large Motion Models with Million-Level Human Motions (ICML 2025 · similarity 0.86)
- Video-Bench: Human-Aligned Video Generation Benchmark (CVPR 2025 · similarity 0.86)
- HuTuMotion: Human-Tuned Navigation of Latent Motion Diffusion Models with Minimal Feedback (AAAI 2024 · similarity 0.86)
- PhyMotion: Structured 3D Motion Reward for Physics-Grounded Human Video Generation (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.