Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
ZeroVO: Visual Odometry with Minimal Assumptions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ZeroVO, a calibration-free transformer-based two-frame monocular visual odometry model with language and depth priors plus geometry-filtered semi-supervised training, evaluated zero-shot on KITTI, nuScenes, Argoverse 2 and a new GTA dataset.
Problem
Monocular visual odometry that generalizes zero-shot across cameras and environments without camera calibration, fine-tuning, or privileged ground-truth supervision, and that stays robust under adverse conditions and drift.
Approach
A cross-attention transformer fuses flow, metric depth, estimated intrinsics (as an intrinsic map), and VLLM-derived language features, with depth and flow unprojected to pseudo-3D scene flow; semi-supervised training filters pseudo-labels using a normalized SSIM geometric consistency check and a subspace-based language similarity check.
Key results
- ZeroVO+ reaches KITTI ATE 104.69 versus 78.53 for DPVO (which uses scale alignment) and 154.77 for M+DS, with nuScenes ATE 9.74 vs 17.08 for M+DS's terr-based comparison; ZeroVO+ achieves the lowest ATE among non-privileged baselines on KITTI, nuScenes, Argoverse 2 and GTA.
- On nuScenes night subset, ZeroVO+ ATE is 10.26 versus 17.19 for M+DS and 14.41 for XVO.
- Inference speed is about 0.6 FPS for ZeroVO+ and 5 FPS for LiteZeroVO+.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the depth module to the flow module (ablation) | ZeroVO · KITTI 00-10, nuScenes, Argoverse, GTA | ATE: 174.24 -> 123.42; 12.54 -> 8.40; 9.42 -> 5.71; 25.38 -> 12.37 | Helped |
| Add the language prior to flow and depth (ablation) | ZeroVO · KITTI 00-10, nuScenes, Argoverse, GTA | ATE: 123.42 -> 105.07; 8.40 -> 6.79; 5.71 -> 4.10; 12.37 -> 8.55 | Helped |
| Add semi-supervised training without pseudo-label selection (ablation) | ZeroVO · KITTI 00-10, nuScenes, Argoverse, GTA | ATE: 105.07 -> 117.49; 6.79 -> 7.53; 4.10 -> 5.13; 8.55 -> 10.27 | Hurt |
| Add pseudo-label selection to semi-supervised training (ablation) | ZeroVO · KITTI 00-10, nuScenes, Argoverse, GTA | ATE: 117.49 -> 104.69; 7.53 -> 6.03; 5.13 -> 3.05; 10.27 -> 8.24 | Helped |
| Replace language-conditioned cross-attention with self-attention (ablation) | ZeroVO+ · KITTI 00-10, nuScenes, Argoverse, GTA | ATE: 104.69 -> 118.54; 6.03 -> 6.87; 3.05 -> 5.28; 8.24 -> 11.26 | Hurt |
Surprising findings
- Adding semi-supervised pseudo-labels without the geometric and language filters degrades performance relative to the supervised model.
Limitations
- Inference is slow (0.6 FPS for ZeroVO+) because the language pipeline depends on LLaVA-NeXT.
- Two-frame formulation drifts over long sequences without global optimization; the paper does not add loop closure or bundle adjustment.
Experimental setup
Datasets: nuScenes YouTube driving videos
Benchmarks: KITTI nuScenes Argoverse 2 GTA
Metrics: translation error rotation error ATE scale error
Baselines: XVO M+DS (Metric3D v2 + Droid-SLAM) DPVO TartanVO
Builds on: TartanVO XVO Metric3Dv2 MaskFlowNet WildCamera
Closest papers by meaning
- Leveraging Consistent Spatio-Temporal Correspondence for Robust Visual Odometry (AAAI 2025 · similarity 0.87)
- LEAP-VO: Long-term Effective Any Point Tracking for Visual Odometry (CVPR 2024 · similarity 0.87)
- MambaVO: Deep Visual Odometry Based on Sequential Matching Refinement and Training Smoothing (CVPR 2025 · similarity 0.84)
- Scene-agnostic Pose Regression for Visual Localization (CVPR 2025 · similarity 0.84)
- One World, Dual Timeline: Decoupled Spatio-Temporal Gaussian Scene Graph for 4D Cooperative Driving Reconstruction (NeurIPS 2026 · similarity 0.84)
- Adaptive VIO: Deep Visual-Inertial Odometry with Online Continual Learning (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.