Research map / 3D vision / Human pose estimation
MotionMap: Representing Multimodality in Human Pose Forecasting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MotionMap, a heatmap-plus-codebook representation of multimodal future human poses that predicts a variable number of modes with confidences, enabling sample-efficient pose forecasting on Human3.6M and AMASS.
Problem
Human pose forecasting is multimodal, but existing stochastic methods need oversampling many predictions to cover modes, and the task is ill-posed as stated; it is unclear which predicted futures are more likely for a given observation.
Approach
Reframe forecasting as recalling transitions present in training data, and represent the distribution of multimodal futures as a 2D heatmap (via t-SNE of future latents) whose local maxima give a variable number of likely modes with confidences; a codebook maps heatmap coordinates back to latents for decoding.
Key results
- On Human3.6M, MotionMap reaches MMADE 0.466 and MMFDE 0.532 vs. BeLFusion's 0.491 and 0.586 under a 7-forecast budget.
- On AMASS, MotionMap achieves ADE 0.325 and MMFDE 0.514, outperforming CoMusion (0.311 ADE is better for CoMusion; MMFDE 0.602) and BeLFusion (MMFDE 0.564).
- MotionMap attains diversity of 7.84 on Human3.6M and 8.50 on AMASS while being sample-efficient.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MotionMap instead of the strongest baseline | Human pose forecasting · Human3.6M | MMADE: 0.491 -> 0.466 | Helped |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · Human3.6M | MMFDE: 0.586 -> 0.532 | Helped |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · Human3.6M | ADE: 0.426 -> 0.474 | Hurt |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · Human3.6M | FDE: 0.597 -> 0.598 | Hurt |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · AMASS | MMADE: 0.488 -> 0.450 | Helped |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · AMASS | MMFDE: 0.564 -> 0.514 | Helped |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · AMASS | ADE: 0.311 -> 0.325 | Hurt |
| Use MotionMap instead of the strongest baseline | Human pose forecasting · AMASS | FDE: 0.460 -> 0.450 | Helped |
Surprising findings
- Several diversity-focused baselines (DLow, DivSamp) are more diverse than the ground truth yet predict less likely futures, so diversity does not imply realism.
Limitations
- Lacks fine-grained motion variation within a mode; MotionMap encodes one forecast per mode, so time-shifted ground truths are clumped.
- Clumping similar motions is a design choice that can raise error on test splits with many time-shifted ground truths.
- Evaluation of multimodal ground truth depends on thresholds (0.5 for Human3.6M, 0.4 for AMASS) and the test split is small, making evaluation unreliable.
Experimental setup
Datasets: Human3.6M AMASS
Benchmarks: Human3.6M AMASS
Metrics: ADE FDE MMADE MMFDE Diversity
Baselines: Zero-Velocity TPK DLow GSPS DivSamp STARS TCD BeLFusion CoMusion
Builds on: BeLFusion DLow t-SNE CoMusion
Closest papers by meaning
- Multimodal Sense-Informed Forecasting of 3D Human Motions (CVPR 2024 · similarity 0.88)
- Expressive Forecasting of 3D Whole-Body Human Motions (AAAI 2024 · similarity 0.88)
- Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning (CVPR 2024 · similarity 0.88)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.88)
- Pose Priors from Language Models (CVPR 2025 · similarity 0.87)
- FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.