Research map / Point clouds / Point cloud classification
DGC-GNN: Leveraging Geometry and Color Cues for Visual Descriptor-Free 2D-3D Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DGC-GNN, a visual descriptor-free 2D-3D matching GNN that fuses global geometric (Euclidean and angular) embeddings, RGB color cues, and cluster-based attention to match 2D keypoints to 3D points.
Problem
Descriptor-free 2D-3D keypoint matching (matching image keypoints to a sparse 3D point cloud without stored visual descriptors) lags far behind descriptor-based methods in accuracy.
Approach
A global-to-local GNN that clusters bearing vectors, encodes global distance and angular geometric relations among clusters, adds per-point RGB color encoding, and uses cluster-restricted attention for local matching, followed by outlier rejection.
Key results
- On MegaDepth with k=1, DGC-GNN reaches 10.2 / 37.64 / 44.04% reprojection AUC at 1/5/10px versus GoMatch's 5.67 / 22.43 / 28.01%.
- On Cambridge Landmarks, DGC-GNN averages 54 cm / 2.23° median error versus GoMatch's 173 cm / 5.87°.
- On 7Scenes, DGC-GNN averages 15 cm / 4.47° median error versus GoMatch's 22 cm / 5.77°.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DGC-GNN instead of GoMatch | Descriptor-free 2D-3D matching · ScanNet, k=1 | Reprojection AUC@1px (%): 1.18 -> 2.73 | Helped |
| Use DGC-GNN instead of GoMatch | Descriptor-free 2D-3D matching · MegaDepth, k=1 | Reprojection AUC@1px (%): 5.67 -> 10.20 | Helped |
| Use DGC-GNN instead of GoMatch | Descriptor-free 2D-3D matching · ScanNet, k=10 | Reprojection AUC@1px (%): 0.91 -> 1.76 | Helped |
| Use DGC-GNN instead of GoMatch | Descriptor-free 2D-3D matching · MegaDepth, k=10 | Reprojection AUC@1px (%): 8.90 -> 15.30 | Helped |
| Use DGC-GNN instead of GoMatch | Visual localization · Cambridge Landmarks | Median translation error (cm): 173 -> 54 | Helped |
| Use DGC-GNN instead of GoMatch | Visual localization · 7Scenes | Median translation error (cm): 22 -> 15 | Helped |
| Add global geometric embedding to the GoMatch pipeline (ablation) | GoMatch pipeline · MegaDepth, k=10 | Reprojection AUC@1px (%): +1.90% | Helped |
| Add cluster attention to the GoMatch pipeline (ablation) | GoMatch pipeline · MegaDepth, k=10 | Reprojection AUC@1px (%): 0.78% | Helped |
| Add color information to the GoMatch pipeline (ablation) | GoMatch pipeline · MegaDepth, k=10 | Reprojection AUC@1px (%): 2.55% | Helped |
| Add angular relation embedding to the GoMatch pipeline (ablation) | GoMatch pipeline · MegaDepth, k=10 | Reprojection AUC@1px (%): 1.11% | Helped |
Surprising findings
- DGC-GNN with k=1 surpasses GoMatch with k=10 on MegaDepth matching AUC.
Limitations
- Performance remains inferior to descriptor-based methods due to insufficient unique 3D structure in geometry.
- Angular embedding is memory-costly (O(Nk'^2)) and therefore omitted in initialization stage.
Experimental setup
Datasets: ScanNet MegaDepth 7Scenes Cambridge Landmarks
Benchmarks: ScanNet MegaDepth 7Scenes Cambridge Landmarks
Metrics: reprojection AUC @1/5/10px rotation error quantiles translation error quantiles matching precision median pose error
Baselines: GoMatch BPnPNet MS-Transformer DSAC* HSCNet HybridSC AS SP+SG
Builds on: GoMatch cluster-based GNN (Cluster-GNN) SuperGlue-style attention BPnPNet
Closest papers by meaning
- SAG-GNN: Semantic-Aware Guided GNN for Descriptor-Free 2D-3D Matching (CVPR 2026 · similarity 0.91)
- PPFNet: Global Context Aware Local Features for Robust 3D Point Matching (CVPR 2018 · similarity 0.88)
- LCD: Learned Cross-Domain Descriptors for 2D-3D Matching (AAAI 2020 · similarity 0.88)
- 3DMatch: Learning Local Geometric Descriptors From RGB-D Reconstructions (CVPR 2017 · similarity 0.87)
- GM-R^2: Generative Matching Learning for Unsupervised Geometric Representation and Registration (CVPR 2026 · similarity 0.87)
- SG-PGM: Partial Graph Matching Network with Semantic Geometric Fusion for 3D Scene Graph Alignment and Its Downstream Tasks (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.