Research map / Detection and segmentation / Tracking
LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LEAP, a long-term point tracker with anchor-based dynamic track labeling, Cauchy-distribution temporal uncertainty, and CoTracker-based training, and uses it as the front-end of LEAP-VO, a visual odometry system for dynamic scenes.
Problem
Visual odometry in dynamic scenes with occlusion and low texture suffers from two-view correspondence methods that ignore temporal context, lack trajectory reliability estimates, and fail to separate static from dynamic points.
Approach
Uses a long-term TAP model (built on CoTracker) augmented with anchor points for inter-track attention to predict per-track dynamic labels, and a multivariate Cauchy distribution with kernel-based scale matrices to output per-point uncertainty; these drive track filtering in a sliding-window bundle adjustment VO pipeline.
Key results
- On MPI Sintel, LEAP-VO reaches ATE 0.037 m versus 0.076 m for DPVO and 0.131 m for DytanVO.
- On Replica, LEAP-VO achieves ATE 0.204 m, better than DPVO (0.257 m) and DROID-SLAM (0.267 m), though ORB-SLAM2 (0.086 m) and DynaSLAM (0.039 m) are better in ATE.
- On TartanAir-Shibuya, LEAP-VO averages ATE 0.0290 m versus 0.0607 m for DytanVO and 0.1238 m for DROID-SLAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LEAP-VO instead of DynaSLAM | Visual odometry · MPI Sintel | ATE (m): 0.131 -> 0.037 | Helped |
| Use LEAP-VO instead of DytanVO | Visual odometry · TartanAir-Shibuya Average | ATE (m): 0.0607 -> 0.0290 | Helped |
| Add dynamic-track filtering to visibility filtering (ablation) | LEAP-VO · MPI Sintel | ATE: 0.118 -> 0.085 | Helped |
| Add confidence filtering to visibility filtering (ablation) | LEAP-VO · MPI Sintel | ATE: 0.118 -> 0.054 | Helped |
| Combine visibility, dynamic-track, and confidence filtering (ablation) | LEAP-VO · MPI Sintel | ATE: 0.118 -> 0.037 | Helped |
| Use uniform rather than distributed image-gradient keypoint sampling (ablation) | LEAP-VO · TartanAir-Shibuya | ATE (m): 0.0290 -> 0.1511 | Hurt |
| Use random rather than distributed image-gradient keypoint sampling (ablation) | LEAP-VO · TartanAir-Shibuya | ATE (m): 0.0290 -> 0.0366 | Hurt |
| Use SIFT rather than distributed image-gradient keypoint sampling (ablation) | LEAP-VO · TartanAir-Shibuya | ATE (m): 0.0290 -> 0.0355 | Hurt |
| Replace LEAP with CoTracker as the tracking front-end (ablation) | Visual odometry · Replica | ATE: 0.204 -> 0.301 | Hurt |
| Replace LEAP with CoTracker as the tracking front-end (ablation) | Visual odometry · MPI Sintel | ATE: 0.037 -> 0.126 | Hurt |
| Replace LEAP with CoTracker as the tracking front-end (ablation) | Visual odometry · TartanAir-Shibuya | ATE: 0.029 -> 0.160 | Hurt |
Surprising findings
- On Replica, LEAP-VO's ATE (0.204 m) is worse than classical ORB-SLAM2 (0.086 m) and DynaSLAM (0.039 m), which the authors attribute to tracking loss and relocalization in those methods.
Limitations
- Relies on a long-term TAP front-end, and the authors note DynaSLAM and ORB-SLAM2 fail on some sequences where LEAP-VO does not, while LEAP-VO itself loses accuracy on Replica relative to classical SLAM.
- Anchor-based dynamic labeling depends on query distribution and count; with a single query no inter-track information is available.
Experimental setup
Datasets: TAP-Vid-Kubric
Benchmarks: Replica MPI Sintel TartanAir-Shibuya
Metrics: ATE RPE trans RPE rot
Baselines: ORB-SLAM2 DynaSLAM DROID-SLAM TartanVO DytanVO DPVO AirDOS CoTracker
Builds on: CoTracker PIPs DROID-SLAM DPVO TAP-Vid-Kubric
Closest papers by meaning
- Beyond Tracking: Selecting Memory and Refining Poses for Deep Visual Odometry (CVPR 2019 · similarity 0.89)
- A Global Occlusion-Aware Approach to Self-Supervised Monocular Visual Odometry (AAAI 2021 · similarity 0.88)
- Leveraging Consistent Spatio-Temporal Correspondence for Robust Visual Odometry (AAAI 2025 · similarity 0.87)
- ZeroVO: Visual Odometry with Minimal Assumptions (CVPR 2025 · similarity 0.87)
- Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (CVPR 2024 · similarity 0.87)
- Is This Tracker On? A Benchmark Protocol for Dynamic Tracking (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.