Research map / Point clouds / Registration and matching
Partial Point Cloud Registration with Multi-view 2D Image Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IAPReg, which projects a point cloud into six multi-view depth maps, encodes them with a pretrained ViT alongside a KPConv 3D encoder, and uses 2D-derived weights in a weighted SVD for partial point cloud registration.
Problem
Partial-to-partial point cloud registration is hard because of outliers and low overlap, and existing methods use only 3D geometry or require extra RGB training images.
Approach
Generate multi-view depth maps from the input point cloud itself (no extra images), fuse 2D ViT features with 3D KPConv features via inter-modality multi-view learning, and use 2D semantic attention scores to weight correspondences in a weighted SVD.
Key results
- On ModelNet40 partial-to-partial registration, IAPReg reaches 96.58% RR with 1.138° RRE, versus 95.72% RR for UDPReg.
- On 3DMatch, IAPReg achieves 93.8% RR and on 3DLoMatch 75.9% RR, state of the art among methods not using extra RGB-D data.
- The 2D encoder adds about 2 points of registration recall in the ablation (93.03% to 95.21%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use IAPReg instead of UDPReg | ModelNet · ModelNet | RR (%): 95.72 -> 96.58 | Helped |
| use IAPReg instead of UDPReg | ModelLoNet · ModelLoNet | RTE (m): 0.069 -> 0.031 | Helped |
| use IAPReg instead of PEAL | 3DMatch · 3DMatch | RR (%): 94.6 -> 93.8 | Hurt |
| use IAPReg instead of PEAL | 3DLoMatch · 3DLoMatch | RR (%): 81.7 -> 75.9 | Hurt |
| add the 2D encoder (ablation) | IAPReg with 3D encoder and 2D semantic score · ModelNet | RR (%): 95.21 -> 96.58 | Helped |
| add the 2D encoder (ablation) | IAPReg with 3D encoder and 2D semantic score · ModelNet | RRE (°): 1.329 -> 1.138 | Helped |
| add the 2D encoder (ablation) | IAPReg with 3D encoder and 2D semantic score · ModelNet | RTE (m): 0.040 -> 0.010 | Helped |
| add the 2D semantic score (ablation) | IAPReg with 3D and 2D encoders · ModelNet | RR (%): 94.67 -> 96.58 | Helped |
| add the 2D semantic score (ablation) | IAPReg with 3D and 2D encoders · ModelNet | RRE (°): 1.669 -> 1.138 | Helped |
| add the 2D semantic score (ablation) | IAPReg with 3D and 2D encoders · ModelNet | RTE (m): 0.039 -> 0.010 | Helped |
| use multi-view projected depth maps instead of native single-view depth maps (ablation) | IAPReg · 3DMatch | RR (%): 90.2 -> 93.8 | Helped |
| use multi-view projected depth maps instead of RGB images (ablation) | IAPReg · 3DMatch | RR (%): 94.7 -> 93.8 | Hurt |
Surprising findings
- Multi-view projected depth maps perform almost as well as RGB images (93.8% vs 94.7% RR on 3DMatch), despite RGB being a real image source.
Limitations
- Relies on a fixed set of six projected views, which may not capture all geometry.
- Gains on 3DLoMatch are less pronounced than on 3DMatch, and some baselines (PEAL, RoITr) score higher on the 3DMatch metrics.
Experimental setup
Datasets: ModelNet40 ModelNet-LoNet 3DMatch 3DLoMatch
Benchmarks: ModelNet40 ModelNet-LoNet 3DMatch 3DLoMatch
Metrics: RR RRE RTE IR FMR
Baselines: ICP DCP-v2 OMNet FINet RPMNet RIENet Predator-50k RegFormer REGTR GeoTransformer UDPReg PEAL
Builds on: KPConv I2P-MAE GeoTransformer PREDATOR
Closest papers by meaning
- FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators (ICLR 2024 · similarity 0.88)
- Generative Point Cloud Registration (ICML 2025 · similarity 0.88)
- Implicit Correspondence Learning for Image-to-Point Cloud Registration (CVPR 2025 · similarity 0.87)
- MM-Point: Multi-View Information-Enhanced Multi-Modal Self-Supervised 3D Point Cloud Understanding (AAAI 2024 · similarity 0.87)
- 3D Focusing-and-Matching Network for Multi-Instance Point Cloud Registration (NeurIPS 2024 · similarity 0.87)
- Multi-View Representation is What You Need for Point-Cloud Pre-Training (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.