Research map / Video / Video generation
Repurposing Video Diffusion Transformers for Cross-View Temporal Object Correspondence
Siyoon Jin, Dahyun Chung, Honggyu An, Sangbeom Lim, Wonjin Nam, Jaewoo Jung, WonJun Moon, Seungryong Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces cross-view temporal object correspondence, which propagates object identity across views and time from a single query mask, and repurposes video diffusion transformers for it.
Problem
Existing cross-view object correspondence methods treat the task as frame-level matching that requires a query mask at every frame, and without temporal context they are ambiguous under appearance changes or occlusions.
Approach
Reframes cross-view correspondence as a temporal problem that preserves object identity over time, and repurposes pretrained video diffusion transformers as the backbone for this task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add focal cross-view attention alignment loss (ablation) | XTrack · Ego-Exo4D Exo2Ego | Ego J&F: 0.303 -> 0.332 | Improved |
| add hard negative conditioning (ablation) | XTrack · Ego-Exo4D Exo2Ego | Ego J&F: 0.332 -> 0.431 | Improved |
| add local context crop for small objects (ablation) | XTrack · Ego-Exo4D Exo2Ego | Ego J&F: 0.431 -> 0.509 | Improved |
| use pretrained video DiT weights (ablation) | XTrack · Ego-Exo4D Exo2Ego | Ego J&F: 0.054 -> 0.509 | Improved |
| add focal attention alignment loss (ablation) | XTrack · Ego-Exo4D Exo2Ego | Exo J&F: 0.415 -> 0.514 | Improved |
| add hard negative conditioning (ablation) | XTrack · Ego-Exo4D Exo2Ego | Exo VA: 0.906 -> 0.882 | Hurt |
| joint unified model vs match-then-track with SAM2 (V2-SAM) | XTrack vs V2-SAM M->T · Ego-Exo4D Ego2Exo | J&F: 0.545 -> 0.472 | Hurt |
| joint view-time prediction instead of decoupled match-then-track | XTrack vs V2-SAM M->T · Ego-Exo4D Ego2Exo | VA: 0.628 -> 0.900 | Improved |
Closest papers by meaning
- Emergent Temporal Correspondences from Video Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- DIVOTrack: A Novel Dataset and Baseline Method for Cross-View Multi-Object Tracking in DIVerse Open Scenes (IJCV 2023 · similarity 0.88)
- Video Diffusion Models Excel at Tracking Similar-Looking Objects Without Supervision (NeurIPS 2025 · similarity 0.88)
- TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking (NeurIPS 2026 · similarity 0.88)
- FRESCO: Spatial-Temporal Correspondence for Zero-Shot Video Translation (CVPR 2024 · similarity 0.87)
- Learning Temporally Consistent Video Depth from Video Diffusion Priors (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
