Research map / 3D vision / 3D generation
Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces scene-aware diverse human motion prediction in real 3D scenes via DiMoP3D, which infers interaction targets from point clouds and past motion, plans obstacle-free trajectories and end-poses, and guides a diffusion generator with them.
Problem
Diverse human motion prediction usually ignores the 3D environment, causing penetration and implausible interactions when applied to real-world scenes.
Approach
Predict a stochastic conditional factor (interaction target, HOI end-pose, obstacle-free trajectory) from the scene and past motion, then use it to self-prompt a diffusion motion generator with a MotionCLIP semantic loss, so diversity is controlled by scene-determined targets.
Key results
- On GIMO, DiMoP3D reaches ADE 5.66 and FDE 6.81 versus BeLFusion's 9.69 and 11.19, with ACPD 0.98 versus 13.73.
- On CIRCLE, DiMoP3D reaches ADE 5.09 and FDE 6.12 versus BeLFusion's 7.91 and 9.30, with ACPD 0.87 versus 14.06.
- DiMoP3D's APD (48.30 on GIMO) is lower than SmoothDMP's 68.80 and DLow's 55.12, so diversity is not the top metric.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DiMoP3D instead of BeLFusion | GIMO motion prediction · GIMO | ACPD: 13.73 -> 0.98 | Helped |
| Use DiMoP3D instead of BeLFusion | CIRCLE motion prediction · CIRCLE | ACPD: 14.06 -> 0.87 | Helped |
| Use DiMoP3D instead of BiFU | GIMO motion prediction · GIMO | ADE: 7.11 -> 5.66 | Helped |
| Use DiMoP3D instead of BiFU | CIRCLE motion prediction · CIRCLE | ADE: 5.80 -> 5.09 | Helped |
| add InterestNet for scene-motion interest analysis (ablation) | DiMoP3D · GIMO | ADE: 6.17 -> 5.66 | Helped |
| add HOI-Estimator for interactive end-pose prediction (ablation) | DiMoP3D · GIMO | ACPD: 1.53 -> 0.98 | Helped |
| add TrajectoryPlanner for obstacle-free paths (ablation) | DiMoP3D · GIMO | ACPD: 3.29 -> 0.98 | Helped |
| add SemanticInspector (ablation) | DiMoP3D · GIMO | ADE: 5.82 -> 5.66 | Helped |
| replace SoftGroup with PointGroup segmentation (ablation) | DiMoP3D · GIMO | ADE: 5.66 -> 5.70 | Hurt |
| replace SoftGroup with HAIS segmentation (ablation) | DiMoP3D · GIMO | ADE: 5.66 -> 5.67 | Hurt |
| use Naive A* instead of the modified trajectory planner (ablation) | DiMoP3D · GIMO | ADE: 5.66 -> 5.93 | Hurt |
| remove the continuous cost model from the trajectory planner (ablation) | DiMoP3D · GIMO | ADE: 5.66 -> 5.77 | Hurt |
Surprising findings
- Naively appending scene features to baselines gives marginal gains, and DLow fails to converge with them.
- Motion synthesis methods show higher ADE than FDE, attributed to poor conditioning on past motion.
Limitations
- Predicts motion at a fixed sequence length; shorter actions are handled by keeping the subject static and longer ones by truncation.
- Relies on pre-existing scene segmentation and interest-map annotations derived from heuristics.
- Diversity (APD) is lower than some non-scene-aware baselines.
Experimental setup
Datasets: GIMO CIRCLE HumanML3D ScanNet
Benchmarks: GIMO CIRCLE
Metrics: APD ADE FDE MMADE MMFDE ACPD FID
Baselines: DLow SmoothDMP BeLFusion BiFU SAMP DN-Synt AffordMotion
Builds on: BeLFusion DLow SoftGroup MotionCLIP SMPL-X
Closest papers by meaning
- Motion Diversification Networks (CVPR 2024 · similarity 0.88)
- Vision-Guided Action: Enhancing 3D Human Motion Prediction with Gaze-informed Affordance in 3D Scenes (CVPR 2025 · similarity 0.88)
- FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations (CVPR 2024 · similarity 0.87)
- Multimodal Sense-Informed Forecasting of 3D Human Motions (CVPR 2024 · similarity 0.87)
- Deterministic-to-Stochastic Diverse Latent Feature Mapping for Human Motion Synthesis (CVPR 2025 · similarity 0.86)
- MotionMap: Representing Multimodality in Human Pose Forecasting (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.