Research map / Probabilistic and scientific modeling / Molecules and proteins
MS-TIP: Imputation Aware Pedestrian Trajectory Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MS-TIP, a pedestrian trajectory prediction framework that jointly imputes missing observed coordinates with a diagonal-masked transformer and predicts endpoints via multiscale hypergraphs, scene attention, and control-point refinement.
Problem
Pedestrian trajectory prediction usually assumes complete observed sequences, but real observations contain missing values from sensor failures and occlusion, which degrades prediction accuracy.
Approach
Combine a non-autoregressive transformer imputation module (diagonal masked self-attention, trained with masked reconstruction losses) with an endpoint-conditioned predictor that uses multiscale KNN hypergraphs for group interactions, VGG-19 scene attention, and a correction refinement module.
Key results
- MS-TIP achieves average ADE/FDE of 0.22/0.36 across ETH, HOTEL, UNIV, ZARA1, ZARA2, outperforming Graph-TERN (the prior endpoint-prediction SOTA).
- Best per-dataset results on ETH, HOTEL, UNIV, ZARA1; second-best on ZARA2.
- Ablation: imputation alone lowers average ADE/FDE from 0.25/0.44 (baseline GCN) to 0.24/0.39; adding scene attention and multiscale hypergraph reaches 0.22/0.36.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MS-TIP instead of G-Tern | ETH pedestrian prediction · ETH | ADE: 0.42 -> 0.39 | Helped |
| Use MS-TIP instead of G-Tern | ETH pedestrian prediction · ETH | FDE: 0.58 -> 0.57 | Helped |
| Use MS-TIP instead of G-Tern | HOTEL pedestrian prediction · HOTEL | ADE: 0.14 -> 0.13 | Helped |
| Use MS-TIP instead of G-Tern | HOTEL pedestrian prediction · HOTEL | FDE: 0.23 -> 0.22 | Helped |
| Add imputation to the simple GCN baseline (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.25/0.44 -> 0.24/0.39 | Helped |
| Add scenic attention to imputation (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.24/0.39 -> 0.23/0.37 | Helped |
| Add multiscale hypergraph to imputation and scenic attention (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.23/0.37 -> 0.22/0.36 | Helped |
| Increase imputation from 15% to 20% (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.23/0.37 -> 0.25/0.39 | Hurt |
| Use uniform rather than variable hyperedge weights at 15% imputation (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.24/0.38 -> 0.22/0.36 | Helped |
| Predict 3 rather than 1 intermediate control point (ablation) | MS-TIP on AVG · AVG | ADE/FDE: 0.26/0.46 -> 0.22/0.36 | Helped |
Surprising findings
- Uniform hyperedge weights outperformed the distance-based variable weighting scheme of Eq. 10 in the ablation.
Limitations
- Missing-data evaluation uses artificially injected NaNs at fixed rates rather than real sensor dropout.
- Per-dataset masking rates were chosen per benchmark (15% for ETH/ZARA2, 5% for others), which is tuned on test results.
- Hypergraph scale set and control point count K were selected by ablation on the test benchmarks.
Experimental setup
Datasets: ETH HOTEL UNIV ZARA1 ZARA2
Benchmarks: ETH HOTEL UNIV ZARA1 ZARA2
Metrics: ADE FDE
Baselines: STGAT CARPE SGCN GroupNet STT S-Implicit MemoNet BCDiff G-Tern FlowChain
Builds on: Graph-TERN GroupNet Gaussian mixture density endpoint prediction (Gupta et al., 2018 / Mangalam et a Self-attention imputation (Du et al., 2023)
Closest papers by meaning
- Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking (NeurIPS 2024 · similarity 0.92)
- ProDiff: Prototype-Guided Diffusion for Minimal Information Trajectory Imputation (ICML 2025 · similarity 0.86)
- Higher-order Relational Reasoning for Pedestrian Trajectory Prediction (CVPR 2024 · similarity 0.86)
- SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents (AAAI 2024 · similarity 0.86)
- GigaTraj: Predicting Long-term Trajectories of Hundreds of Pedestrians in Gigapixel Complex Scenes (CVPR 2024 · similarity 0.86)
- TrajCLIP: Pedestrian trajectory prediction method using contrastive learning and idempotent networks (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.