Research map / Label-efficient and robust learning / Recognition and long-tail

Towards Generalizable Trajectory Prediction using Dual-Level Representation Learning and Adaptive Prompting

CVPR 2025 · Label-efficient and robust learning › Recognition and long-tail · approach: Contrastive representation theory · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PerReg+, a Perceiver IO-based vehicle trajectory predictor pretrained with self-distillation and masked reconstruction on multiple datasets, then adapted with frozen-backbone prompt tuning.

Problem

Vehicle trajectory prediction models generalize poorly across datasets, handle multimodal uncertainty inefficiently (clustering, NMS, large trajectory pools), and do not exploit self-supervised pretraining with Perceiver-style architectures.

Approach

Combines teacher-student self-distillation with segment-level masked reconstruction (with query dropping) in a Perceiver IO model, uses register queries alongside mode queries to avoid NMS/clustering, and fine-tunes by freezing the network and optimizing cluster-selected prompts.

Key results

• Pre-training reduces B-FDE by 11% on nuScenes and 13% on AV2 versus PerReg without pretraining in single-dataset training (PerReg+ B-FDE 2.62 on nuScenes).
• Cross-domain test (train WOMD, test nuScenes) gives PerReg+ B-FDE 2.75 vs 3.12 for non-pretrained PerReg, an 11.8% reduction.
• Ablation: register queries reduce B-FDE by 11% over the baseline; the full model reaches B-FDE 2.62 on nuScenes.

Measured effects reported in the paper

Surprising findings

• Pre-training gives only a small gain on the large WOMD dataset (2.4%) versus larger gains on small datasets.
• Forecast-MAE gains only minor improvements from pretraining in out-of-domain tests (about 0.6% B-FDE).

Limitations

• Pretraining benefit is smaller on the large WOMD dataset.
• Evaluation restricted to vehicle trajectories and the UniTraj framework.

Experimental setup

Datasets: nuScenes Argoverse 2 Waymo Open Motion Dataset

Benchmarks: nuScenes Argoverse 2 Waymo Open Motion Dataset

Metrics: B-FDE minADE minFDE MR

Baselines: AutoBot MTR Forecast-MAE QCNet

Builds on: Perceiver IO UniTraj Forecast-MAE Wayformer DWA

Closest papers by meaning

• GTR: A General, Multi-View, and Dynamic Framework for Trajectory Representation Learning (ICML 2025 · similarity 0.86)
• T4P: Test-Time Training of Trajectory Prediction via Masked Autoencoder and Actor-specific Token Memory (CVPR 2024 · similarity 0.85)
• Trajectory Mamba: Efficient Attention-Mamba Forecasting Model Based on Selective SSM (CVPR 2025 · similarity 0.84)
• Prioritizing Perception-Guided Self-Supervision: A New Paradigm for Causal Modeling in End-to-End Autonomous Driving (NeurIPS 2025 · similarity 0.84)
• PLMTrajRec: A Scalable and Generalizable Trajectory Recovery Method with Pre-trained Language Models (NeurIPS 2025 · similarity 0.84)
• Grounding Driving VLA via Inverse Kinematics (NeurIPS 2026 · similarity 0.84)