Research map / 3D vision / Depth and stereo
DELTA: DENSE EFFICIENT LONG-RANGE 3D TRACKING FOR ANY VIDEO
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DELTA, a feed-forward transformer that tracks every pixel in 3D over long monocular RGB-D videos via reduced-resolution global-local spatial attention plus an attention-based upsampler, and finds log-depth representation best.
Problem
Dense 3D tracking of every pixel over long monocular videos is computationally prohibitive for existing point-tracking methods, which use cross-track attention and sparse tracks.
Approach
Coarse-to-fine tracking: run a CoTracker-style transformer at 1/4 resolution with joint global (anchor-track cross-attention for virtual tracks) and dense local spatial attention, then upsample with an attention-based upsampler using Alibi distance bias; additionally represent depth as log depth ratios log(d_t/d_1).
Key results
- On Kubric-3D (24 frames), DELTA reaches AJ 81.4 and APD_3D 88.6 vs DOT-3D's 72.3 and 77.5, running in 0.5 min.
- On TAP-Vid3D average with UniDepth, DELTA achieves AJ 13.1 and APD_3D 20.6, the best among compared methods.
- On CVO-Extended, the 2D variant reaches EPE 3.63 vs DOT's 4.98 and CoTracker's 5.20; log-depth representation improves TAP-Vid3D AJ from 9.0 (depth) to 13.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use log-depth representation and predict log(d_t/d_1) instead of Euclidean or inverse depth (ablation) | DELTA · TAP-Vid3D average | AJ: 9.0 -> 13.1 | Helped |
| Use log-depth representation and predict log(d_t/d_1) instead of Euclidean or inverse depth (ablation) | DELTA · TAP-Vid3D average | APD_3D: 15.0 -> 20.6 | Helped |
| Add local attention to anchor-track global attention (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 3.73/ 2.78 / 5.47 -> 3.67 / 2.64 / 5.30 | Helped |
| Replace CoTracker global attention with anchor-track global attention (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 3.72 / 2.78 / 5.44 -> 3.73/ 2.78 / 5.47 | No clear effect |
| Add global attention to local attention only (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 8.01/ 3.89 / 13.91 -> 3.67 / 2.64 / 5.30 | Helped |
| Add both global and local attention rather than neither (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 10.0 / 4.84 / 18.1 -> 3.67 / 2.64 / 5.30 | Helped |
| Use attention-based upsampling rather than ConvUp (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 4.27 / 3.09 / 6.73 -> 3.73 / 2.73 / 5.35 | Helped |
| Add Alibi to AttentionUp (ablation) | DELTA · CVO-Extended | EPE all/vis/occ: 3.73 / 2.73 / 5.35 -> 3.67 / 2.64 / 5.30 | Helped |
| Add anchor tracks (ablation) | DELTA (2D) · CVO-Extended | EPE all/vis/occ: 4.50 / 3.10 / 6.95 -> 3.63 / 2.67 / 5.24 | Helped |
| Add 3D supervision | DELTA · CVO-Clean, CVO-Final, and CVO-Extended | EPE all/vis/occ: 0.89 / 0.46 / 2.96 -> 0.94 / 0.51 / 2.97; 0.97 / 0.55 / 2.96 -> 1.03 / 0.61 / 3.03; 3.63 / 2.67 / 5.24 -> 3.67 / 2.64 / | Mixed |
| Fine-tune on the LSFOdyssey training set | DELTA · LSFOdyssey | AJ: 29.4 -> 50.1 | Helped |
| Use DELTA rather than DOT-3D | 3D dense tracking · Kubric3D | AJ: 72.3 -> 81.4 | Helped |
Surprising findings
- Better 2D tracking models do not always yield better 3D tracking (e.g., BootsTAPIR underperforms CoTracker in 3D lifting), attributed to scale inconsistency of frame-wise depth estimators.
Limitations
- Fails to track points occluded for extended periods due to short temporal windows.
- Performs best on videos with fewer than a few hundred frames.
- 3D tracking accuracy depends on the quality and temporal consistency of off-the-shelf monocular depth.
Experimental setup
Datasets: Kubric (generated) LSFOdyssey
Benchmarks: CVO Kubric-3D TAP-Vid3D LSFOdyssey TAP-Vid
Metrics: EPE IoU AJ APD_3D OA APD_2D
Baselines: RAFT MFT TAPIR CoTracker DOT SceneTracker SpatialTracker DOT-3D
Builds on: CoTracker RAFT LocoTrack SpatialTracker
Closest papers by meaning
- TrackingWorld: World-centric Monocular 3D Tracking of Almost All Pixels (NeurIPS 2025 · similarity 0.90)
- TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking (NeurIPS 2026 · similarity 0.89)
- TAPVid-3D: A Benchmark for Tracking Any Point in 3D (NeurIPS 2024 · similarity 0.88)
- TAPIP3D: Tracking Any Point in Persistent 3D Geometry (NeurIPS 2025 · similarity 0.88)
- SpatialTracker: Tracking Any 2D Pixels in 3D Space (CVPR 2024 · similarity 0.87)
- FlowTrack: Revisiting Optical Flow for Long-Range Dense Tracking (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.