Research map / 3D vision / Depth and stereo
3D Feature Tracking via Event Camera
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the first 3D feature tracker that fuses stereo event streams with a joint 2D-offset and disparity model (motion compensation, bi-polarity hypergraph patch matching, stereo motion consistency), and releases the E-3DTrack dataset with 250 FPS 3D ground truth.
Problem
Tracking 3D trajectories of features on high-speed moving objects, where frame-based cameras miss motion and no event-based 3D feature tracking dataset with ground truth exists.
Approach
Jointly predict 2D feature motion offsets and disparity from stereo event patches, using template deformation compensation, bi-polarity hypergraph aggregation for patch matching, and a stereo motion consistency loss linking offsets and disparity change.
Key results
- On E-3DTrack the method reaches RMSE 0.1181 and TFR(0.1m) 0.6928, versus 0.1889 RMSE for the best baseline DeepEvT + SDE.
- Inference time is 40.30 ms per step, a 56.8% reduction versus DeepEvT + SDE (93.22 ms).
- Ablations show removing the bi-polarity hypergraph module raises RMSE by 20.8%, and removing stereo motion consistency raises it by 11.1%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use joint 3D feature tracking instead of DeepEvT + SDE | E-3DTrack · E-3DTrack test | RMSE: 0.1889 -> 0.1181 | Helped |
| use joint 3D feature tracking instead of DeepEvT + SDE | E-3DTrack · E-3DTrack test | TFR (0.1m): 0.5782 -> 0.6928 | Helped |
| use joint 3D feature tracking instead of DeepEvT + SDE | E-3DTrack · E-3DTrack test | time (ms/step): 93.22 -> 40.30 | Helped |
| add BiHCM to the base model (ablation) | E-3DTrack model · E-3DTrack test | TFR (0.1m): 0.5586 -> 0.6082 | Helped |
| add stereo motion consistency to the base model (ablation) | E-3DTrack model · E-3DTrack test | RMSE: 0.1807 -> 0.1512 | Helped |
| add motion compensation to the base model (ablation) | E-3DTrack model · E-3DTrack test | RMSE: 0.1807 -> 0.1624 | Helped |
| add motion compensation to BiHCM and stereo motion consistency (ablation) | E-3DTrack model · E-3DTrack test | RMSE: 0.1268 -> 0.1181 | Helped |
| add stereo motion consistency to BiHCM and motion compensation (ablation) | E-3DTrack model · E-3DTrack test | RMSE: 0.1312 -> 0.1181 | Helped |
| add BiHCM to stereo motion consistency and motion compensation (ablation) | E-3DTrack model · E-3DTrack test | RMSE: 0.1427 -> 0.1181 | Helped |
Surprising findings
- Simply chaining 2D event feature trackers with stereo depth estimators gives poor long-term 3D tracking, indicating that 2D trajectory and 3D position must be modeled jointly.
Limitations
- Evaluated only on a single self-collected dataset with 10 test scenarios.
- Tracking relies on an initial template patch and is restricted to local patch-based matching.
Experimental setup
Datasets: E-3DTrack
Benchmarks: E-3DTrack
Metrics: Feature Age Tracked Feature Ratio RMSE inference time
Baselines: E-RAFT + TSES E-RAFT + SDE EKLT + TSES EKLT + SDE DeepEvT + TSES DeepEvT + SDE
Builds on: DeepEvT HGNN+ EKLT E-RAFT
Closest papers by meaning
- ETAP: Event-based Tracking of Any Point (CVPR 2025 · similarity 0.84)
- TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels (NeurIPS 2025 · similarity 0.83)
- SpatialTracker: Tracking Any 2D Pixels in 3D Space (CVPR 2024 · similarity 0.83)
- TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking (NeurIPS 2026 · similarity 0.83)
- Active Event-based Stereo Vision (CVPR 2025 · similarity 0.82)
- Event Stream-based Visual Object Tracking: A High-Resolution Benchmark Dataset and A Novel Baseline (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.