Research map / Label-efficient and robust learning / Recognition and long-tail
Towards Predicting Any Human Trajectory In Context
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces TrajICL, an in-context learning framework for pedestrian trajectory prediction that adapts to new scenes without weight updates, using spatio-temporal similarity example selection, prediction-guided refinement, and synthetic MOTSynth training.
Problem
Pedestrian trajectory prediction models generalize poorly across scenes and domains, and fine-tuning per scenario is impractical on edge devices.
Approach
Adapt a trajectory predictor at inference by conditioning on in-context examples from the same scene, selected by spatio-temporal similarity and then refined using the model's own predicted future trajectories (prediction-guided example selection), trained on large-scale synthetic data.
Key results
- On MOTSynth, TrajICL reaches 15.3/17.5 minADE/minFDE versus 16.0/20.9 for the best fine-tuned method (Full FT), without any fine-tuning.
- With only 10% of the example pool, TrajICL improves minFDE by 8.8% on MOTSynth and 11.9% on JTA over the best fine-tuning method.
- PG-ES improves over STES by 8.1% on MOTSynth and 8.6% on SDD in minFDE; on JTA, TrajICL reduces minFDE by 56.9% relative to vanilla Social-Transmotion.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · MOTSynth | minFDE_K: 23.0 -> 17.5 | Helped |
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · JRDB-Image | minFDE_K: 3.32 -> 2.68 | Helped |
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · WildTrack | minFDE_K: 36.3 -> 28.3 | Helped |
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · SDD | minFDE_K: 18.9 -> 14.8 | Helped |
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · JRDB-World | minFDE_K: 0.26 -> 0.21 | Helped |
| Use TrajICL instead of vanilla Social-Transmotion | Social-Transmotion · JTA | minFDE_K: 1.97 -> 0.85 | Helped |
| Add prediction-guided selection to STES (ablation) | TrajICL · MOTSynth | minFDE_K: 19.2 -> 17.5 | Helped |
| Add prediction-guided selection to STES (ablation) | TrajICL · WildTrack | minFDE_K: 32.5 -> 28.3 | Helped |
| Add prediction-guided selection to STES (ablation) | TrajICL · JTA | minFDE_K: 0.91 -> 0.85 | Helped |
| Add prediction-guided selection to STES (ablation) | TrajICL · SDD | minFDE_K: 16.2 -> 14.8 | Helped |
| Add both spatial and temporal similarity to random example selection (ablation) | TrajICL · MOTSynth, WildTrack, JTA, and SDD | minFDE_K: 21.8 -> 17.5; 35.5 -> 28.3; 1.08 -> 0.85; 16.4 -> 14.8 | Helped |
| Add first-stage VTP training (ablation) | TrajICL · MOTSynth, WildTrack, and JTA | minFDE_K: 17.6 -> 17.5; 30.4 -> 28.3; 0.89 -> 0.85 | Helped |
Surprising findings
- Random example selection gives minimal gains and even no improvement on JTA as the number of in-context examples grows.
- On JRDB-World, TrajICL is slightly worse than fine-tuned methods in minADE (0.13 vs 0.09-0.10).
Limitations
- Inference cost is higher than the base model due to the predictor over in-context examples and two-stage PG-ES.
- Synthetic training data has limited motion diversity; generalization relies on in-scene example pool at test time.
- Gains are smaller or absent on some datasets (e.g., JRDB-World minADE, NBA rebounding at 10% pool).
Experimental setup
Datasets: MOTSynth JRDB WildTrack SDD JTA ETH-UCY NBA SportVU
Benchmarks: MOTSynth JRDB-Image JRDB-World WildTrack SDD JTA ETH-UCY NBA SportVU
Metrics: minADE_K minFDE_K
Baselines: Social-Transmotion Full fine-tuning LoRA VPT Head tuning ForecastMAE Latent Corridors RAN MetaTra
Builds on: Social-Transmotion in-context learning MOTSynth
Closest papers by meaning
- TrajCLIP: Pedestrian trajectory prediction method using contrastive learning and idempotent networks (NeurIPS 2024 · similarity 0.89)
- Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking (NeurIPS 2024 · similarity 0.88)
- Social-Transmotion: Promptable Human Trajectory Prediction (ICLR 2024 · similarity 0.87)
- SocialCVAE: Predicting Pedestrian Trajectory via Interaction Conditioned Latents (AAAI 2024 · similarity 0.86)
- GigaTraj: Predicting Long-term Trajectories of Hundreds of Pedestrians in Gigapixel Complex Scenes (CVPR 2024 · similarity 0.86)
- A Unified Environmental Network for Pedestrian Trajectory Prediction (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.