Research map / Point clouds / Registration and matching
Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes B2-3Dnet, a detection-free image-to-point cloud registration network with uncertainty-modeled multi-scale image patches (UHMM) and adversarial domain alignment via a gradient reversal layer (AMAM), evaluated on RGB-D Scenes V2 and 7-Scenes.
Problem
Image-to-point cloud registration suffers from uniform attention to noisy image patches and a large domain gap between dense 2D images and sparse 3D point clouds.
Approach
Model per-patch uncertainty of multi-scale image features (variance-constrained by an entropy loss) so that critical patches get more weight, hierarchically fuse image patch features into point cloud patches, and align image and point cloud feature domains with a GRL-based domain classifier used only during training.
Key results
- On RGB-D Scenes V2, mean registration recall 63.4% vs 56.4% for 2D3D-MATR, with inlier ratio 35.1 vs 32.4.
- On 7-Scenes, mean registration recall 77.7% vs 75.8% for 2D3D-MATR; mean inlier ratio 50.9 vs 50.1.
- Feature matching recall improves to 94.4% (RGB-D Scenes V2) vs 90.8% for 2D3D-MATR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · RGB-D Scenes V2 test (mean) | Inlier Ratio: 32.4 -> 35.1 | Helped |
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · RGB-D Scenes V2 test (mean) | Feature Matching Recall: 90.8 -> 94.4 | Helped |
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · RGB-D Scenes V2 test (mean) | Registration Recall: 56.4 -> 63.4 | Helped |
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · 7-Scenes test (mean) | Inlier Ratio: 50.1 -> 50.9 | Helped |
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · 7-Scenes test (mean) | Feature Matching Recall: 92.1 -> 93.1 | Helped |
| add B2-3Dnet's uncertainty-aware hierarchical matching and domain alignment | 2D3D-MATR · 7-Scenes test (mean) | Registration Recall: 75.8 -> 77.7 | Helped |
| add the interaction stage without the uncertainty estimation layer (ablation) | B2-3Dnet baseline · RGB-D Scenes V2 | Inlier Ratio | Helped |
| add the interaction stage without the uncertainty estimation layer (ablation) | B2-3Dnet baseline · RGB-D Scenes V2 | Registration Recall | Hurt |
| add the uncertainty estimation layer to the interaction stage (ablation) | B2-3Dnet · RGB-D Scenes V2 | Registration Recall | Helped |
| add AMAM to the interaction-stage variant (ablation) | B2-3Dnet · RGB-D Scenes V2 | Inlier Ratio | Helped |
Limitations
- Uncertainty is modeled only for image patches, not point cloud patches, by design choice rather than evaluation.
- Gains over 2D3D-MATR on 7-Scenes are small (about 0.8 to 2 pp).
Experimental setup
Datasets: RGB-D Scenes V2 7-Scenes
Benchmarks: RGB-D Scenes V2 7-Scenes
Metrics: Inlier Ratio Feature Matching Recall Registration Recall
Baselines: FCGF-2D3D P2-Net Predator-2D3D 2D3D-MATR
Builds on: 2D3D-MATR circle loss Gradient Reversal Layer KPFCNN ResNet FPN
Closest papers by meaning
- Generative Point Cloud Registration (ICML 2025 · similarity 0.89)
- Implicit Correspondence Learning for Image-to-Point Cloud Registration (CVPR 2025 · similarity 0.87)
- Multi-scale Consistency for Robust 3D Registration via Hierarchical Sinkhorn Tree (NeurIPS 2024 · similarity 0.87)
- FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators (ICLR 2024 · similarity 0.86)
- A Consistency-Aware Spot-Guided Transformer for Versatile and Hierarchical Point Cloud Registration (NeurIPS 2024 · similarity 0.86)
- Zero-shot RGB-D Point Cloud Registration with Pre-trained Large Vision Model (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.