Research map / Detection and segmentation / Tracking
TAPIR: Tracking Any Point with Per-Frame Initialization and Temporal Refinement
Carl Doersch, Yi Yang, Mel Vecerik, Dilara Gokay, Ankush Gupta, Yusuf Aytar, Joao Carreira, Andrew Zisserman
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add iterative refinement (4 iterations vs none) (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 41.6 -> 61.3 | Improved |
| add depthwise convolution instead of chunked MLP-Mixer (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 53.8 -> 61.3 | Improved |
| add stride-4 high-resolution features (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 54.0 -> 61.3 | Improved |
| add uncertainty estimate (ablation) | TAPIR · TAP-Vid-Kinetics | AJ: 54.4 -> 57.2 | Improved |
| add TAP-Net per-frame initialization instead of constant initialization (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 59.3 -> 61.3 | Improved |
| combine TAP-Net initialization with PIPs-style refinement (simple combination) (ablation) | Chain-Free PIPs · TAP-Vid-DAVIS | AJ: 43.3 -> 50.0 | Improved |
| train on Panning MOVi-E instead of MOVi-E (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 59.8 -> 61.3 | Improved |
| train on Panning MOVi-E instead of MOVi-E (ablation) | TAPIR · TAP-Vid-RGB-Stacking | AJ: 66.2 -> 62.7 | Hurt |
| increase refinement iterations from 4 to 6 (ablation) | TAPIR · TAP-Vid-DAVIS | AJ: 61.3 -> 60.4 | Hurt |
| run TAPIR at original high resolution with image pyramid | TAPIR · TAP-Vid-DAVIS | AJ / occlusion accuracy: AJ 61.3 -> 65.7; OA 88.8 -> 86.7 | Mixed |
| use tuned open-source hyperparameters, instance norm and no max-pool | TAPIR · TAP-Vid DAVIS and Kinetics | AJ: approximately 1% | Improved |
| use more than 3 feature pyramid levels (ablation) | TAPIR · TAP-Vid | AJ | No clear effect |
Later papers that compared with this method
5 later accepted papers report TAPIR as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- TAPNext: Tracking Any Point (TAP) as Next Token Prediction (ICCV 2025 · better on 4 of 4 shared results)
- Local All-Pair Correspondence for Point Tracking (ECCV 2024 · better on 7 of 7 shared results)
- DriveTrack: A Benchmark for Long-Range Point Tracking in Real-World Videos (CVPR 2024 · better on 5 of 6 shared results)
- CoTracker: It is Better to Track Together (ECCV 2024 · better on 4 of 5 shared results)
- TAPTR: Tracking Any Point with Transformers as Detection (ECCV 2024 · better on 2 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- TAPVid-3D: A Benchmark for Tracking Any Point in 3D (NeurIPS 2024 · similarity 0.90)
- TAPVid-360: Tracking Any Point in 360 from Narrow Field of View Video (NeurIPS 2025 · similarity 0.89)
- TAPTRv2: Attention-based Position Update Improves Tracking Any Point (NeurIPS 2024 · similarity 0.88)
- TAPIP3D: Tracking Any Point in Persistent 3D Geometry (NeurIPS 2025 · similarity 0.88)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.88)
- TrackIME: Enhanced Video Point Tracking via Instance Motion Estimation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
