Research map / Label-efficient and robust learning / Recognition and long-tail
TrajCLIP: Pedestrian trajectory prediction method using contrastive learning and idempotent networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TrajCLIP, which aligns historical and future trajectory encoders via CLIP-style contrastive learning and maps history to future features with an idempotent generative network (IGN) for pedestrian trajectory prediction.
Problem
Pedestrian trajectory prediction where existing generative methods map historical-trajectory feature spaces to separate future-trajectory feature spaces, creating a gap that hurts cross-scene generalization and adaptation to new scenes.
Approach
Use contrastive learning (CLIP-style pairing of historical and future trajectory encoders, with a velocity-direction similarity matrix as labels) to put history and future in the same feature space, then use an idempotent generative network with idempotent and tightness losses to map history features to future features via an affine-like MLP, combined with frequency-domain scene-agent interaction
Key results
- On ETH-UCY, SDD and SNU, TrajCLIP reports average ADE/FDE of 0.18/0.33 on ETH-UCY (vs. 0.21/0.32 for SingularTrajectory) and 6.29/11.79 on SDD, the best among compared methods.
- In scene-to-scene transfer on ETH-UCY, TrajCLIP achieves average ADE/FDE of 0.68/1.36, better than T-GNN (0.80/1.56) and K0 (0.77/1.51).
- Ablation shows removing CLIP pairing with IGN raises ETH-UCY error by 0.06/0.17 relative to the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TrajCLIP instead of the strongest listed baseline | ETH-UCY AVG · ETH-UCY AVG | ADE: 0.21 -> 0.18 | Helped |
| Use TrajCLIP instead of the strongest listed baseline | ETH-UCY AVG · ETH-UCY AVG | FDE: 0.32 -> 0.33 | Mixed |
| Use TrajCLIP instead of the strongest listed baseline | SDD · SDD | ADE: 7.26 -> 6.29 | Helped |
| Use TrajCLIP instead of the strongest listed baseline | SDD · SDD | FDE: 12.58 -> 11.79 | Helped |
| Use TrajCLIP instead of the strongest listed baseline | SNU · SNU | ADE: 0.28 -> 0.25 | Helped |
| Use TrajCLIP instead of the strongest listed baseline | SNU · SNU | FDE: 0.42 -> 0.39 | Helped |
| Add SAIF (ablation) | TrajCLIP · ETH-UCY | ADE: 0.20 -> 0.18 | Helped |
| Add SAIF (ablation) | TrajCLIP · SDD | ADE: 6.34 -> 6.29 | Helped |
| Add CLIP pairing learning to IGN (ablation) | TrajCLIP · ETH-UCY | ADE: 0.24 -> 0.18 | Helped |
| Add CLIP pairing learning to IGN (ablation) | TrajCLIP · SDD | ADE: 7.85 -> 6.29 | Helped |
| Replace IGN with CVAE, retaining CLIP (ablation) | TrajCLIP · ETH-UCY | ADE: 0.18 -> 0.23 | Hurt |
| Replace IGN with diffusion, retaining CLIP (ablation) | TrajCLIP · ETH-UCY | ADE: 0.18 -> 0.21 | Hurt |
Surprising findings
- Adding the CLIP alignment gives little benefit to CVAE and diffusion generators, and helps mainly with the IGN MLP generator.
- Applying the proposed data augmentation to EqMotion and TUTR yields minimal gains, while TrajCLIP improves significantly.
Limitations
- Fails to ensure collision avoidance in complex, high-density environments.
- Interpolation may reduce accuracy despite improving capacity.
Experimental setup
Datasets: ETH-UCY SDD SNU
Benchmarks: ETH-UCY SDD SNU
Metrics: ADE FDE
Baselines: Trajectron++ AgentFormer MID GP-Graph EqMotion TUTR SingularTrajectory T-GNN K0 tra2tra MemoNet Social-STGCNN
Builds on: CLIP IGN (idempotent generative network) AgentFormer CVAE diffusion models
Closest papers by meaning
- Towards Predicting Any Human Trajectory In Context (NeurIPS 2025 · similarity 0.89)
- TSC-Net: Prediction of Pedestrian Trajectories by Trajectory-Scene-Cell Classification (ICLR 2025 · similarity 0.89)
- Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking (NeurIPS 2024 · similarity 0.88)
- A Unified Environmental Network for Pedestrian Trajectory Prediction (AAAI 2024 · similarity 0.87)
- Generative Human Trajectory Recovery via Embedding-Space Conditional Diffusion (ICML 2025 · similarity 0.86)
- MS-TIP: Imputation Aware Pedestrian Trajectory Prediction (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.