Research map / Detection and segmentation / Tracking
Exploring Temporally-Aware Features for Point Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Chrono, a DINOv2-based feature backbone with a temporal adapter between transformer blocks, enabling accurate point tracking via plain soft-argmax correlation matching without a refiner.
Problem
Point tracking methods rely on simple backbones trained from scratch on synthetic data and on costly per-query iterative refiners to inject temporal information; temporally-aware pre-trained feature backbones remain underexplored.
Approach
Insert lightweight temporal adapters (strided 2D conv plus local 1D temporal attention with residual) between every block of a frozen-style pre-trained DINOv2 so features themselves carry temporal context, then track by soft-argmax over cosine correlation with no learnable post-feature layers.
Key results
- Chrono (ViT-B/14) reaches 70.1 average position accuracy on TAP-Vid-DAVIS strided mode versus 54.4 for DINOv2 ViT-B/14 and 49.2 for TSM-ResNet-18 (Table 1).
- Chrono (ViT-B/14) achieves 12.5x the throughput of TAPIR with only a 3.5-point drop on DAVIS (70.1 vs 73.6).
- Chrono adapted into LocoTrack improves AJ on RGB-Stacking by 6.1 points (77.1 to 83.2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Chrono (ViT-B/14) instead of ResNet-18 features | Point tracking · TAP-Vid RGB-Stacking strided | < δx avg: 73.9 -> 86.0 | Helped |
| Use Chrono (ViT-B/14) instead of CoTraker Backbone features | Point tracking · TAP-Vid Kinetics strided | < δx avg: 58.2 -> 68.5 | Helped |
| Use Chrono (ViT-B/14) instead of CoTraker Backbone features | Point tracking · TAP-Vid DAVIS strided | < δx avg: 58.0 -> 70.1 | Helped |
| Use Chrono (ViT-B/14) instead of ResNet-18 features | Point tracking · TAP-Vid RGB-Stacking first query | < δx avg: 69.6 -> 82.1 | Helped |
| Use Chrono (ViT-B/14) instead of ResNet-18 features | Point tracking · TAP-Vid Kinetics first query | < δx avg: 50.3 -> 61.6 | Helped |
| Use Chrono (ViT-B/14) instead of DINOv2 (ViT-B/14) features | Point tracking · TAP-Vid DAVIS first query | < δx avg: 50.3 -> 65.6 | Helped |
| Replace 1D attention with 1D convolution for temporal aggregation (ablation) | Chrono (ViT-S/14) · TAP-Vid DAVIS | < δx avg: 68.0 -> 65.9 | Hurt |
| Replace 1D attention with 3D convolution for temporal aggregation (ablation) | Chrono (ViT-S/14) · TAP-Vid DAVIS | < δx avg: 68.0 -> 66.4 | Hurt |
| Place six temporal adapters in the early six blocks rather than all blocks (ablation) | Chrono (ViT-S/14) · TAP-Vid DAVIS | < δx avg: 68.0 -> 61.7 | Hurt |
| Place six temporal adapters in the later six blocks rather than all blocks (ablation) | Chrono (ViT-S/14) · TAP-Vid DAVIS | < δx avg: 68.0 -> 65.8 | Hurt |
| Place six temporal adapters in alternating blocks rather than all blocks (ablation) | Chrono (ViT-S/14) · TAP-Vid DAVIS | < δx avg: 68.0 -> 65.9 | Hurt |
Surprising findings
- Chrono features with plain soft-argmax matching and no refiner are comparable to refiner-based pipelines such as TAPIR on DAVIS.
Limitations
- Inference is slower than vanilla DINOv2 for feature extraction (0.575s vs 0.194s for ViT-S on 24 frames).
- Occlusion prediction is not addressed; the method focuses on position accuracy only.
Experimental setup
Datasets: Kubric Panning-MOVi-E
Benchmarks: TAP-Vid-DAVIS TAP-Vid-Kinetics TAP-Vid-RGB-Stacking
Metrics: position accuracy (delta_x avg) average Jaccard (AJ) occlusion accuracy throughput inference time GPU memory
Baselines: ResNet-18 (TAPIR) TSM-ResNet-18 (TAP-Net) CoTracker backbone DINOv2 ViT-S/14 DINOv2 ViT-B/14 RAFT PIPs TAPIR LocoTrack FlowTrack
Builds on: DINOv2 TAP-Net TAPIR LocoTrack TSM
Closest papers by meaning
- Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better (CVPR 2025 · similarity 0.88)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.87)
- TrackIME: Enhanced Video Point Tracking via Instance Motion Estimation (NeurIPS 2024 · similarity 0.85)
- ETAP: Event-based Tracking of Any Point (CVPR 2025 · similarity 0.85)
- No More Shortcuts: Realizing the Potential of Temporal Self-Supervision (AAAI 2024 · similarity 0.85)
- Is This Tracker On? A Benchmark Protocol for Dynamic Tracking (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.