Research map / Detection and segmentation / Open-set and few-shot detection
MATCHA: Towards Matching Anything
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MATCHA, a single unified feature descriptor for geometric, semantic and temporal correspondence, built by attention-based fusion of Stable Diffusion (DIFT) features with DINOv2 features under correspondence supervision.
Problem
Establishing correspondences across images for geometric, semantic, and temporal matching typically requires specialized, task-specific descriptors; unsupervised foundation features like DIFT still need per-task feature selection and lag supervised methods in accuracy.
Approach
An attention-based dynamic fusion module (self- and cross-attention) mutually enhances SD semantic and geometric features under correspondence supervision (CLIP contrastive + dense semantic flow loss for semantic, dual softmax for geometric), then statically concatenates the result with DINOv2 features into one 1024-d descriptor.
Key results
- MATCHA achieves the highest average score (79.6) across geometric (Aachen AUC@5/10/20 avg 60.4), semantic (PF-Willow PCK avg 86.2) and temporal (TAP-Vid-Davis PCK avg 92.3) matching, versus DIFT at 72.1.
- On PF-Willow, MATCHA reaches PCK@0.05 of 70.2 versus 58.7 for unsupervised DIFT.Uni+DINO and 55.7 for DIFT.
- On relative pose estimation, MATCHA+SP obtains AUC@5/10/20 of 55.8/69.3/80.0 on MegaDepth, improving over DIFT by about 6 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dynamic feature fusion to the supervised DIFT descriptors (ablation) | MATCHA-Light · Aachen | AUC@5/10/20: 50.4 / 58.7 / 65.7 -> 51.4 / 60.1 / 67.1 | Helped |
| Add dynamic feature fusion to the supervised DIFT descriptors (ablation) | MATCHA-Light · PF-Willow | PCK@0.05/0.1/0.15: 32.7 / 46.4 / 55.6 -> 69.0 / 90.6 / 96.2 | Helped |
| Add semantic correspondence supervision to the DIFT semantic descriptor (ablation) | DIFT.S · PF-Willow | PCK@0.05/0.1/0.15: 55.7 / 85.1 / 92.9 -> 63.6 / 88.4 / 95.7 | Helped |
| Add semantic correspondence supervision to the DIFT semantic descriptor (ablation) | DIFT.S · Aachen | AUC@5/10/20: 25.6 / 35.6 / 46.3 -> 11.5 / 18.6 / 27.7 | Hurt |
| Add DINOv2 features to the fused, supervised DIFT descriptors (ablation) | M1 · Aachen | AUC@5/10/20: 21.9 / 31.4 / 41.3 -> 29.2 / 39.5 / 49.7 | Helped |
| Add DINOv2 features to the fused, supervised DIFT descriptors (ablation) | M1 · PF-Willow | PCK@0.05/0.1/0.15: 69.0 / 90.6 / 96.2 -> 70.3 / 92.4 / 97.6 | Helped |
| Use a unified descriptor instead of separate semantic and geometric descriptors (ablation) | MATCHA · PF-Willow | PCK@0.05/0.1/0.15: 53.1 / 76.8 / 85.5 -> 70.2 / 91.3 / 97.0 | Helped |
| Use a unified descriptor instead of separate semantic and geometric descriptors (ablation) | MATCHA · Aachen | AUC@5/10/20: 54.0 / 62.7 / 69.8 -> 51.7 / 61.0 / 68.5 | Hurt |
| Directly concatenate raw DIFT geometric and semantic features without feature fusion (ablation) | DIFT.Uni · PF-Willow | PCK@0.05/0.1/0.15: 55.7 / 85.1 / 92.9 -> 26.4 / 40.4 / 50.6 | Hurt |
| Use a unified descriptor with supervision but without feature fusion or DINOv2 (ablation) | M2 · PF-Willow | PCK@0.05/0.1/0.15: 63.6 / 88.4 / 95.7 -> 31.8 / 45.6 / 55.4 | Hurt |
| Add DINOv2 to DIFT.Uni (ablation) | DIFT.Uni + DINOv2 · Aachen | AUC@5/10/20: 43.6 / 52.7 / 60.8 -> 41.9 / 51.3 / 60.0 | Hurt |
| Add DINOv2 to DIFT.Uni (ablation) | DIFT.Uni + DINOv2 · PF-Willow | PCK@0.05/0.1/0.15: 26.4 / 40.4 / 50.6 -> 58.7 / 82.9 / 90.7 | Helped |
Surprising findings
- DINOv2, despite poor geometric matching, achieves surprisingly strong temporal matching among unsupervised methods, which the authors attribute to handling large viewpoint and scale changes on single dominant objects.
Limitations
- Features derived from foundation models face resolution precision issues for fine-grained geometric matching due to 8x downsampling.
- Not optimized for runtime efficiency.
- Semantic supervision uses limited annotated data, and the approach relies on nearest-neighbor matching rather than dedicated matchers.
Experimental setup
Datasets: SPair-71k PF-Pascal PF-Willow HPatches ScanNet MegaDepth Aachen Day-Night TAP-Vid
Benchmarks: SPair-71k PF-Pascal PF-Willow HPatches ScanNet MegaDepth Aachen Day-Night TAP-Vid DAVIS
Metrics: PCK mean matching accuracy AUC of pose error average ranking score
Baselines: DIFT DINOv2 SD+DINO GeoASM SD4Match DHF DISK XFeat SuperPoint R2D2 D2-Net MASt3R encoder
Builds on: DIFT DINOv2 SD+DINO dual softmax matching (XFeat) CLIP contrastive loss
Closest papers by meaning
- OmniGlue: Generalizable Feature Matching with Foundation Model Guidance (CVPR 2024 · similarity 0.87)
- Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence (ICLR 2024 · similarity 0.87)
- Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching (ICLR 2024 · similarity 0.87)
- ETO:Efficient Transformer-based Local Feature Matching by Organizing Multiple Homography Hypotheses (NeurIPS 2024 · similarity 0.86)
- CoMatcher: Multi-View Collaborative Feature Matching (CVPR 2025 · similarity 0.86)
- RoMa: Robust Dense Feature Matching (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.