Research map / Vision-language and multimodal / Human and embodied modeling
OOSTraj: Out-of-Sight Trajectory Prediction With Vision-Positioning Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OOSTraj, predicting future visual trajectories of out-of-sight pedestrians from noisy sensor (GPS/odometry) trajectories, using an unsupervised vision-positioning denoising module that estimates camera matrices from in-view agents.
Problem
Trajectory prediction for pedestrians or vehicles completely out of camera view, where only noisy mobile sensor trajectories (GPS, odometry) are available and no ground truth for denoised sensor data exists.
Approach
Uses precise visual trajectories of in-view agents to estimate camera matrix embeddings, projects denoised sensor trajectories of out-of-sight agents into camera coordinates via a world-to-camera transform, and uses the projected visual trajectory as unsupervised denoising supervision, followed by a transformer predictor in the visual modality.
Key results
- On Vi-Fi, the proposed method reaches SUM 27.24 (MSE-D 13.42, MSE-P 13.83), versus 28.33 for Vanilla Transformer and 200.90 for ViTag.
- On JRDB, the method reaches SUM 25.51 (MSE-D 10.52, MSE-P 14.99), versus 33.37 for Vanilla Transformer.
- Plug-and-play VPD on Vanilla LSTM reduces Vi-Fi SUM from 118.44 to 30.39.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OOSTraj instead of Vanilla Transformer | Out-of-sight trajectory denoising and prediction · Vi-Fi | SUM: 28.33 -> 27.24 | Helped |
| use OOSTraj instead of Vanilla Transformer | Out-of-sight trajectory denoising and prediction · JRDB | SUM: 33.37 -> 25.51 | Helped |
| add Camera Parameters Estimator (CPE) (ablation) | OOSTraj · Vi-Fi | SUM: 32.12 -> 27.24 | Helped |
| add Camera Parameters Estimator (CPE) (ablation) | OOSTraj · JRDB | SUM: 29.66 -> 25.51 | Helped |
| add Mobile Denoising Encoder (MDE) (ablation) | OOSTraj · Vi-Fi | SUM: 32.33 -> 27.24 | Helped |
| add Mobile Denoising Encoder (MDE) (ablation) | OOSTraj · JRDB | SUM: 32.94 -> 25.51 | Helped |
| add Visual Positioning Projection (VPP) (ablation) | OOSTraj · Vi-Fi | SUM: 28.42 -> 27.24 | Helped |
| add Visual Positioning Projection (VPP) (ablation) | OOSTraj · JRDB | SUM: 32.48 -> 25.51 | Helped |
| add Out-of-Sight Prediction Decoder (OPD) (ablation) | OOSTraj · Vi-Fi | SUM: 27.33 -> 27.24 | Helped |
| add Out-of-Sight Prediction Decoder (OPD) (ablation) | OOSTraj · JRDB | SUM: 30.01 -> 25.51 | Helped |
Surprising findings
- ViTag, the state-of-the-art visual-wireless association method on Vi-Fi, underperforms simple baselines on the out-of-sight trajectory task.
- Vanilla RNN performs well on Vi-Fi but poorly on JRDB (SUM 112.40), attributed to differing sensor noise characteristics.
Limitations
- Assumes the camera is static within a window or that camera matrices can be estimated from in-view agents, which may fail for fast-moving cameras.
- Out-of-sight evaluation is simulated by masking one visible pedestrian per sequence rather than real out-of-sight data.
- Requires in-view agents with both visual and sensor trajectories to estimate camera parameters.
Experimental setup
Datasets: Vi-Fi Multimodal Dataset JackRabbot Dataset (JRDB)
Benchmarks: Vi-Fi JRDB
Metrics: MSE-T (MSE-D) MSE-T (MSE-P) SUM
Baselines: ViTag Vanilla LSTM Vanilla GRU Vanilla RNN Vanilla Transformer
Builds on: Transformer geometric camera calibration ViTag
Closest papers by meaning
- C2F-TP: A Coarse-to-Fine Denoising Framework for Uncertainty-Aware Trajectory Prediction (AAAI 2025 · similarity 0.84)
- OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection (NeurIPS 2025 · similarity 0.84)
- Pedestrian Trajectory Prediction with Missing Data: Datasets, Imputation, and Benchmarking (NeurIPS 2024 · similarity 0.84)
- Certified Human Trajectory Prediction (CVPR 2025 · similarity 0.84)
- GigaTraj: Predicting Long-term Trajectories of Hundreds of Pedestrians in Gigapixel Complex Scenes (CVPR 2024 · similarity 0.84)
- ZeroVO: Visual Odometry with Minimal Assumptions (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.