Research map / 3D vision / 3D generation
Three-Dimensional Trajectory Prediction with 3DMoTraj Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the 3DMoTraj dataset of UUV 3D trajectories with intention labels and a decoupled-then-correlated 3D trajectory predictor (LBEBM baseline plus SCA-LSTM refinement) that beats prior 2D methods on it.
Problem
Predicting future trajectories in 3D is harder than 2D (more parameters per point), and no public dataset existed for 3D trajectory prediction.
Approach
Decompose 3D trajectory prediction into three independent per-axis predictions (decoupled prediction, built on LBEBM's key-point decoders) and then refine them with an LSTM-based SCA-LSTM that models inter-axis correlations via state-correlation and state-aggregation modules.
Key results
- On 3DMoTraj (best-of-20), the method reaches mean ADE/FDE 0.58/1.02, versus 0.68/1.22 for the second-best S-Implicit.
- Decoupled prediction improves over the LBEBM decoder by 19.0% in ADE and FDE in the ablation.
- On 2D benchmarks, the method improves over the LBEBM baseline by 4.5%/5.0% on ETH/UCY and 2.4%/3.3% on SDD (ADE/FDE).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the LBEBM decoder with independent per-axis decoders (ablation) | LBEBM on 3DMoTraj · 3DMoTraj | ADE: 0.84 -> 0.68 | Helped |
| add trajectory refinement with vanilla LSTM (ablation) | Decoupled prediction on 3DMoTraj · 3DMoTraj | ADE: 0.68 -> 0.66 | Helped |
| replace vanilla LSTM refinement with SCA-LSTM (ablation) | Decoupled prediction on 3DMoTraj · 3DMoTraj | ADE: 0.66 -> 0.58 | Helped |
| replace vanilla LSTM refinement with SCA-LSTM (ablation) | Decoupled prediction on 3DMoTraj · 3DMoTraj | FDE: 1.05 -> 1.02 | Helped |
| set SC and SA iteration count to 1 instead of 3 (ablation) | SCA-LSTM on 3DMoTraj · 3DMoTraj | ADE: 0.58 -> 0.62 | Hurt |
| set SC and SA iteration count to 2 instead of 3 (ablation) | SCA-LSTM on 3DMoTraj · 3DMoTraj | ADE: 0.58 -> 0.67 | Hurt |
| set SC and SA iteration count to 4 instead of 3 (ablation) | SCA-LSTM on 3DMoTraj · 3DMoTraj | FDE: 1.02 -> 1.07 | Hurt |
| increase SCA-LSTM depth from 1 to 2 layers (ablation) | SCA-LSTM on 3DMoTraj · 3DMoTraj | ADE: 0.58 -> 0.64 | Hurt |
| increase SCA-LSTM depth from 1 to 2 layers (ablation) | SCA-LSTM on 3DMoTraj · 3DMoTraj | FDE: 1.02 -> 1.07 | Hurt |
| use the proposed method instead of LBEBM | 2D trajectory prediction · ETH/UCY | ADE: 0.22 -> 0.21 | Helped |
| use the proposed method instead of LBEBM | 2D trajectory prediction · SDD | ADE: 9.20 -> 8.98 | Helped |
Surprising findings
- Using 3 SCA-LSTM iterations is best, but a single SCA-LSTM layer is best while stacking more layers degrades performance.
Limitations
- Struggles with trajectories containing multiple sharp bends over short intervals (failure case).
- Uses simple interaction modeling and no 3D point cloud maps.
- Dataset covers only UUVs in underwater scenarios with three agents per scenario.
Experimental setup
Datasets: 3DMoTraj ETH/UCY SDD
Benchmarks: 3DMoTraj ETH/UCY SDD
Metrics: ADE FDE
Baselines: SSTGCNN MSRL FlowChain PECNet LBEBM NPSN TrajCLIP CausalHTP MS-TIP MRGTraj S-Implicit
Builds on: LBEBM MLP key point prediction vanilla LSTM
Closest papers by meaning
- Lifting Motion to the 3D World via 2D Diffusion (CVPR 2025 · similarity 0.84)
- Trajectory World Models for Heterogeneous Environments (ICML 2025 · similarity 0.84)
- MolmoMotion: Forecasting Point Trajectories in 3D with Language Instruction (NeurIPS 2026 · similarity 0.83)
- Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos (CVPR 2025 · similarity 0.83)
- PointZero: 3D Point Track Completion for Learning Transferable 3D Dynamics (NeurIPS 2026 · similarity 0.83)
- Harmonizing Stochasticity and Determinism: Scene-responsive Diverse Human Motion Prediction (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.