Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Cross-View Completion Models are Zero-shot Correspondence Estimators
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that the cross-attention map of CroCo-v2 encodes dense correspondence better than encoder/decoder features and builds zero-shot (ZeroCo) and learned matching and multi-frame depth models on top of it.
Problem
Understanding why cross-view completion pretraining yields strong geometric features, and which internal representation of CroCo-v2 best encodes correspondence for dense matching and depth tasks.
Approach
Draws an analogy between cross-view completion and self-supervised correspondence learning, finds the decoder cross-attention map (pre-softmax, with reciprocal swapped-input fusion) is a better matching cost than encoder or decoder correlations, and uses it zero-shot and with lightweight learnable heads.
Key results
- ZeroCo zero-shot on HPatches-240 reaches 9.41 average AEPE versus 35.84 for CroCo encoder+decoder correlation and 26.14 for DIFT.
- ZeroCo-flow achieves 13.61 average AEPE on HPatches-Original (learning-based), outperforming MASt3R's 17.01 and other listed dense matchers.
- ZeroCo-depth reaches AbsRel 0.090 on KITTI Eigen split with two frames, with lower SqRel (0.637) than DepthFormer and DualRefine among the listed methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use cross-attention maps instead of encoder or decoder correlations as matching costs (ablation) | CroCo-v2 · HPatches-240 | AEPE: 47.52 -> 10.85; 44.63 -> 10.85 | Helped |
| enforce reciprocity by incorporating swapped inputs (ablation) | ZeroCo cross-attention matching · HPatches-240 | AEPE: 10.85 -> 10.24 | Helped |
| apply L2 normalization without reciprocity (ablation) | Cross-attention matching · HPatches-240 | AEPE: 10.85 -> 10.66 | Helped |
| apply softmax normalization without reciprocity (ablation) | Cross-attention matching · HPatches-240 | AEPE: 10.85 -> 10.53 | Helped |
| apply L2 normalization when reciprocity is enforced (ablation) | Reciprocal cross-attention matching · HPatches-240 | AEPE: 10.24 -> 10.34 | Hurt |
| apply softmax normalization when reciprocity is enforced (ablation) | Reciprocal cross-attention matching · HPatches-240 | AEPE: 10.24 -> 10.45 | Hurt |
| add dense zoom-in (ablation) | Reciprocal cross-attention matching · HPatches-240 | AEPE: 10.24 -> 9.41 | Helped |
| use ZeroCo-flow rather than the strongest competing method | Dense geometric matching · HPatches-Original | AEPE: 18.84 -> 13.61 | Helped |
| use ZeroCo-flow rather than MASt3R | Dense geometric matching · ETH3D | AEPE: 2.62 -> 2.88 | Hurt |
| use ZeroCo-depth instead of DualRefine | Multi-frame depth estimation · KITTI Eigen split | AbsRel: 0.087 -> 0.090 | Hurt |
| use ZeroCo-depth instead of DynamicDepth | Dynamic-object depth estimation · Cityscapes | AbsRel: 0.143 -> 0.127 | Helped |
Surprising findings
- Decoder feature correlations fail to find correct correspondences while the cross-attention map localizes them sharply, despite DUSt3R/MASt3R relying on decoder descriptors.
- Normalizing the cross-attention map (L2-norm or softmax) degrades performance once reciprocity is enforced.
Limitations
- Learned variants rely on lightweight heads and cost aggregation; zero-shot cross-attention is limited for fine-grained matching due to implicit learning.
- ZeroCo-finetuned underperforms on HPatches-Original relative to ZeroCo-flow, and ZeroCo-flow is not best on all ETH3D rates versus MASt3R.
Experimental setup
Datasets: HPatches ETH3D COCO DPED-CityScapes-ADE MegaDepth KITTI Cityscapes
Benchmarks: HPatches ETH3D KITTI Eigen split Cityscapes dynamic objects
Metrics: AEPE AbsRel SqRel RMSE delta1 mDEE mRR
Baselines: DINOv2 DIFT SD-DINO CroCo encoder/decoder correlations DGC-Net GLU-Net-GOCor PDCNet+ DiffMatch DUSt3R MASt3R DepthFormer DynamicDepth
Builds on: CroCo-v2 DUSt3R MASt3R DIFT ManyDepth
Closest papers by meaning
- Understanding Multi-View Transformers (NeurIPS 2026 · similarity 0.87)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.85)
- DeCoTR: Enhancing Depth Completion with 2D and 3D Attentions (CVPR 2024 · similarity 0.84)
- Self-Supervised Cross-View Correspondence with Predictive Cycle Consistency (CVPR 2025 · similarity 0.84)
- CoMatcher: Multi-View Collaborative Feature Matching (CVPR 2025 · similarity 0.84)
- C3Po: Cross-View Cross-Modality Correspondence by Pointmap Prediction (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.