Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Multi-agent Long-term 3D Human Pose Forecasting via Interaction-aware Trajectory Conditioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes T2P, a coarse-to-fine interaction-aware model that forecasts multi-modal global hip trajectories and then conditions local 3D poses on each mode, and builds the JRDB-GMP long-term multi-agent 3D pose forecasting dataset from JRDB.
Problem
Long-term (up to 5-6s) multi-agent 3D human pose forecasting is hard because multi-modal human motion and inter-agent interactions are poorly modeled, and existing datasets are short and have few agents, so long-term multi-agent evaluation was lacking.
Approach
Decompose human motion into a global hip trajectory and local pose; forecast multi-modal global trajectories first, then condition local pose forecasts on each trajectory mode, with graph-based agent-wise interaction modeling at the trajectory scale (Traj-Pose module) rather than joint-wise across all agents.
Key results
- On JRDB-GMP at 5s horizon, T2P achieves JPE 390.4 vs 481.3 for TBIFormer and 538.8 for JRT (Table 2).
- T2P reports over 10% FDE gain on all datasets; e.g., CMU-Mocap 2s FDE 188.9 vs 228.5 for TBIFormer.
- Interaction modeling at trajectory scale reduces complexity from O(T^2 N^2 J^2) to O(T N^2) and improves JPE@5s on JRDB-GMP by 3.84% (406.0 to 390.4).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Trajectory2Pose instead of JRT | multi-agent pose forecasting · JRDB-GMP, 2s/5s | JPE @ 5s: 538.8 -> 390.4 | Helped |
| Use Trajectory2Pose instead of TBIFormer | multi-agent pose forecasting · JRDB-GMP, 2s/5s | APE @ 5s: 102.9 -> 94.7 | Helped |
| Use Trajectory2Pose instead of JRT | multi-agent pose forecasting · JRDB-GMP, 2s/5s | FDE @ 5s: 497.2 -> 361.0 | Helped |
| Use Trajectory2Pose instead of TBIFormer | multi-agent pose forecasting · 3DPW, 1s/2s | APE @ 2s: 83.3 -> 83.3 | No clear effect |
| Add local pose embedding (ablation) | Trajectory2Pose · JRDB-GMP | JPE @ 5s: 471.4 -> 400.5 | Helped |
| Add agent interaction (ablation) | Trajectory2Pose · JRDB-GMP | JPE @ 5s: 406.0 -> 390.4 | Helped |
| Add trajectory-conditioning (ablation) | Trajectory2Pose pose decoder · JRDB-GMP | APE @ 5s: 95.2 -> 91.4 | Helped |
| Increase prediction modes from 1 to 6 (ablation) | Trajectory2Pose · CMU-Mocap (UMPM), 1s/2s | APE @ 2s: 154.4 -> 151.7 | Helped |
| Increase prediction modes from 1 to 6 (ablation) | Trajectory2Pose · CMU-Mocap (UMPM), 1s/2s | JPE @ 2s: 366.4 -> 262.7 | Helped |
| Widen interaction range from < 2m to < 4m (ablation) | Trajectory2Pose · JRDB-GMP | JPE @ 5s: 403.5 -> 400.5 | Helped |
| Widen interaction range from < 4m to all agents (ablation) | Trajectory2Pose · JRDB-GMP | JPE @ 5s: 400.5 -> 390.4 | Helped |
Surprising findings
- Baseline TBIFormer improves only 0.52% from body-part-wise interaction modeling, while the trajectory-scale interaction modeling improves T2P by 3.84%.
Limitations
- JRDB-GMP poses are extracted with a monocular 3D pose estimator (BEV) and filtered using 2D annotations, so ground truth may be noisy.
- Global trajectory prediction relies on a HiVT-based baseline encoder structure.
Experimental setup
Datasets: CMU-Mocap (UMPM) 3DPW JRDB
Benchmarks: CMU-Mocap (UMPM) 3DPW JRDB-GMP
Metrics: JPE APE FDE
Baselines: MRT JRT TBIFormer HiVT
Builds on: HiVT TBIFormer MRT JRDB
Closest papers by meaning
- FutureHuman3D: Forecasting Complex Long-Term 3D Human Behavior from Video Observations (CVPR 2024 · similarity 0.88)
- MotionMap: Representing Multimodality in Human Pose Forecasting (CVPR 2025 · similarity 0.88)
- Multi-modal Knowledge Distillation-based Human Trajectory Forecasting (CVPR 2025 · similarity 0.87)
- Expressive Forecasting of 3D Whole-Body Human Motions (AAAI 2024 · similarity 0.87)
- Multimodal Sense-Informed Forecasting of 3D Human Motions (CVPR 2024 · similarity 0.87)
- SimpliHuMoN: Simplifying Human Motion Prediction (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.