Research map / Image generation / Autoregressive and latent generation
OmniGlue: Generalizable Feature Matching with Foundation Model Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniGlue, a sparse learnable image matcher that uses frozen DINOv2 features to prune inter-image attention graphs and a position-guided attention that disentangles keypoint positions from descriptors, improving zero-shot generalization to novel image domains.
Problem
Learnable feature matchers reach strong in-domain accuracy but generalize poorly to novel image domains (object-centric, aerial), often falling behind classical SIFT.
Approach
Use DINOv2 similarity to guide which keypoints participate in inter-image cross-attention, and compute attention weights from descriptor+position while updating only descriptor components, so positional priors do not contaminate the matching descriptors.
Key results
- On six novel domains OmniGlue gives 20.9% average relative gain over SuperGlue and 9.5% over LightGlue on pose and registration accuracy at tightest thresholds.
- On MegaDepth-1500 OmniGlue reaches 47.4 AUC@5 vs LightGlue 47.6 and SuperGlue 42.2.
- Low-shot fine-tuning on 30 GSO sneaker instances raises Hard-subset pose accuracy@5 by 94.2% relative over zero-shot base.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OmniGlue instead of LightGlue | SuperPoint-based matching · MegaDepth-1500 | AUC@5°: 47.6 -> 47.4 | No clear effect |
| use OmniGlue instead of the strongest listed baseline | image matching · GSO-Hard | pose accuracy @5°: 7.6 -> 8.6 | Helped |
| use OmniGlue instead of the strongest listed baseline | image matching · GSO-Easy | pose accuracy @5°: 38.2 -> 38.4 | No clear effect |
| use OmniGlue instead of SuperGlue | image matching · NAVI-Wild | pose accuracy @5°: 10.6 -> 12.4 | Helped |
| use OmniGlue instead of LightGlue | image matching · ScanNet | pose accuracy @5°: 36.7 -> 31.3 | Hurt |
| use OmniGlue instead of LightGlue | image matching · DeepAerial | PCK@1%: 18.1 -> 22.4 | Helped |
| remove position guidance (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 10.0 | Hurt |
| remove positional embeddings from all attention (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 9.1 | Hurt |
| remove positional embeddings from cross-attention (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 9.3 | Hurt |
| remove DINO guidance (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 9.8 | Hurt |
| merge DINO features with SuperPoint descriptors (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 7.8 | Hurt |
| apply DINO guidance to intra- and inter-image graphs (ablation) | OmniGlue · GSO-Hard | pose accuracy @5°: 11.0 -> 10.5 | Hurt |
Surprising findings
- Removing positional embeddings entirely improves cross-domain generalization despite lowering in-domain performance, suggesting SuperGlue's positional usage hurts transfer.
- Injecting DINOv2 features by merging them with SuperPoint descriptors degrades performance, while using DINO only for graph pruning helps.
Limitations
- Generalization gains are measured on a fixed set of seven datasets; performance on truly unseen domains is not established.
- DINO features are coarse, so the method cannot fully resolve fine-grained matches where DINO similarity is wrong.
- Fine-tuning gains require annotated target-domain correspondences.
Experimental setup
Datasets: Synthetic Homography (Oxford and Paris) MegaDepth Google Scanned Objects NAVI ScanNet DeepAerialMatching
Benchmarks: MegaDepth-1500 Google Scanned Objects (Hard/Easy) NAVI (Multiview/Wild) ScanNet DeepAerialMatching Synthetic Homography SH100/SH200
Metrics: pose AUC@5/10/20 pose accuracy @5/10/20 deg PCK@1%/3%/5% correspondence precision/recall
Baselines: SIFT SuperPoint+MNN SuperGlue LightGlue LoFTR PDCNet DINOv2+SuperGlue
Builds on: SuperGlue SuperPoint DINOv2 LightGlue Sinkhorn algorithm
Closest papers by meaning
- GIM: Learning Generalizable Image Matcher From Internet Videos (ICLR 2024 · similarity 0.88)
- MATCHA: Towards Matching Anything (CVPR 2025 · similarity 0.87)
- Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching (ICLR 2024 · similarity 0.86)
- Unifying Feature and Cost Aggregation with Transformers for Semantic and Visual Correspondence (ICLR 2024 · similarity 0.86)
- General Object Foundation Model for Images and Videos at Scale (CVPR 2024 · similarity 0.85)
- Matching While Perceiving: Enhance Image Feature Matching with Applicable Semantic Amalgamation (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.