Research map / 3D vision / 3D generation
GIM: Learning Generalizable Image Matcher From Internet Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GIM, a self-training framework that learns a single generalizable image matcher from 50 hours of internet videos via multi-method matching, correspondence propagation, and strong augmentation, and introduces the ZEB zero-shot matching benchmark.
Problem
Learning-based image matchers generalize poorly to in-the-wild images because they are trained separately per scene type (indoor/outdoor) on limited, costly-to-build datasets such as ScanNet and MegaDepth.
Approach
Self-train any matching architecture on unlabeled internet videos: generate dense labels on nearby frames by fusing the trained model with complementary matchers, filter them with robust fitting, propagate correspondences to distant frames, and train with strong perspective augmentation, avoiding SfM/MVS pipelines.
Key results
- GIM_DKM improves zero-shot ZEB AUC@5 from 46.2 (DKM in) to 49.4, and to 51.2 with 100 hours of video.
- GIM_LoFTR raises ZEB AUC@5 from 33.1 to 39.1 (+18.1% relative), and GIM_SuperGlue from 31.2 to 34.3 (+9.9% relative).
- Ablation: removing video entirely drops GIM_DKM to 46.2, while removing label propagation yields 47.1 and COLMAP labels yield only 46.5 at matched compute.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply GIM video self-training to SuperGlue | SuperGlue · ZEB | Mean AUC@5° (%): 31.2 → 34.3 | Helped |
| Apply GIM video self-training to LoFTR | LoFTR · ZEB | Mean AUC@5° (%): 33.1 → 39.1 | Helped |
| Apply GIM video self-training to DKM | DKM · ZEB | Mean AUC@5° (%): 46.2 → 49.4 | Helped |
| Increase video training data from 50 hours to 100 hours | GIM DKM · ZEB | Mean AUC@5° (%): 49.4 → 51.2 | Helped |
| Use 25 hours rather than 50 hours of video (ablation) | GIM DKM · ZEB | AUC@5° (%): 49.4 → 48.3 | Hurt |
| Use 12.5 hours rather than 50 hours of video (ablation) | GIM DKM · ZEB | AUC@5° (%): 49.4 → 47.4 | Hurt |
| Use only RootSIFT to generate video labels (ablation) | GIM DKM · ZEB | AUC@5° (%): 49.4 → 49.3 | Hurt |
| Add data augmentation (ablation) | GIM DKM · ZEB | AUC@5° (%): 49.2 → 49.4 | Helped |
| Add label propagation (ablation) | GIM DKM · ZEB | AUC@5° (%): 47.1 → 49.4 | Helped |
| Use GIM video labels instead of COLMAP labels (ablation) | GIM DKM · ZEB | AUC@5° (%): 46.5 → 49.4 | Helped |
| Add video training data (ablation) | GIM DKM · ZEB | AUC@5° (%): 46.2 → 49.4 | Helped |
Surprising findings
- Hand-crafted RootSIFT performs better than or on par with in-domain learning-based models on several ZEB subsets such as GL3, BLE, KIT and GTA.
- Using only RootSIFT to generate video labels reduces performance only slightly (49.3 vs 49.4), suggesting diversity of images matters more than label quality.
Limitations
- Gains on in-domain benchmarks are small because in-domain baselines already overfit to their domains.
- Label generation depends on the base matchers and complementary methods; propagation requires at least 1024 correspondences per pair.
- Performance on some ZEB subsets (e.g., WEA, NIG, MUL) remains close to baselines or only modestly improved.
Experimental setup
Datasets: MegaDepth ScanNet 50 hours of YouTube tourism videos GL3D BlendedMVS ETH3D KITTI RobotcarWeather RobotcarSeason RobotcarNight Multi-FoV SceneNet RGB-D
Benchmarks: ZEB MegaDepth-1500 ScanNet-1500 HPatches Aachen-Day-Night v1.1 InLoc
Metrics: AUC@5 degrees pose error mean rank homography corner error AUC localization accuracy at (0.25m,2deg)/(0.5m,5deg)/(1.0m,10de
Baselines: RootSIFT SuperGlue (in) SuperGlue (out) LoFTR (in) LoFTR (out) DKM (in) DKM (out) SGP
Builds on: SuperGlue LoFTR DKM COLMAP RootSIFT
Closest papers by meaning
- OmniGlue: Generalizable Feature Matching with Foundation Model Guidance (CVPR 2024 · similarity 0.88)
- FC-GNN: Recovering Reliable and Accurate Correspondences from Interferences (CVPR 2024 · similarity 0.86)
- MATCHA: Towards Matching Anything (CVPR 2025 · similarity 0.85)
- XFeat: Accelerated Features for Lightweight Image Matching (CVPR 2024 · similarity 0.84)
- Matcher: Segment Anything with One Shot Using All-Purpose Feature Matching (ICLR 2024 · similarity 0.84)
- FastJAM: a Fast Joint Alignment Model for Images (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.