Research map / Point clouds / Registration and matching
Generative Point Cloud Registration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Generative Point Cloud Registration, using a matching-specific ControlNet variant (Match-ControlNet) to generate depth-conditioned, cross-view texture-consistent RGB image pairs from point clouds, which are fused with geometric descriptors to improve registration on 3DMatch and ScanNet.
Problem
Geometry-only point cloud registration struggles under low overlap, repetitive patterns, and noise because no RGB data is available to supply color cues.
Approach
Generate pairwise RGB images for source and target point clouds with a ControlNet conditioned on depth maps, using coupled conditional denoising (joint latent of both views in one denoising pass) and a coupled prompt, all zero-shot, then fuse the generated color features with geometric descriptors (zero-shot feature fusion or XYZ-RGB fusion).
Key results
- Generative FCGF (Stable Diffusion features) improves 45°-rotation accuracy on ScanNet from 87.5 to 94.4 over FCGF.
- Generative Predator improves ScanNet 45° rotation accuracy by 6.1 points (82.6 to 88.7).
- Generative ColorPCR with generated color on 3DMatch reaches 83.6 at 5° rotation vs 79.9 for ColorPCR with real color.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Stable Diffusion generated-image features to FCGF | FCGF · ScanNet test | Rotation accuracy @45°: 87.5 -> 94.4 | Helped |
| Add Stable Diffusion generated-image features to FCGF | FCGF · 3DMatch | Rotation accuracy @45°: 94.8 -> 98.1 | Helped |
| Replace real color point clouds with generated color point clouds | ColorPCR · 3DMatch | Rotation accuracy @45°: 88.9 -> 93.2 | Helped |
| Add cross-view texture consistency to geometric consistency generation (ablation) | Generative FCGF SD · 3DMatch | Rotation accuracy @45°: 97.8 -> 98.1 | Helped |
| Fine-tune Match-ControlNet rather than use its zero-shot version (ablation) | Generative FCGF SD · 3DMatch | Rotation accuracy @45°: 97.3 -> 98.1 | Helped |
| Fine-tune Match-ControlNet with 1000 samples instead of 3000 (ablation) | Generative FCGF SD · 3DMatch | Rotation accuracy @45°: 98.1 -> 98.1 | No clear effect |
| Fine-tune Match-ControlNet with 5000 samples instead of 3000 (ablation) | Generative FCGF SD · 3DMatch | Rotation accuracy @45°: 98.1 -> 98.0 | No clear effect |
| Set color feature dimension to 16 instead of 64 (ablation) | Generative GeoTrans · 3DMatch | Rotation accuracy @45°: 96.2 -> 96.3 | No clear effect |
| Set color feature dimension to 128 instead of 64 (ablation) | Generative GeoTrans · 3DMatch | Rotation accuracy @45°: 96.2 -> 96.0 | No clear effect |
| Set fusion weight ω to 0.0 instead of 0.50 (ablation) | Generative GeoTrans · 3DMatch | Rotation accuracy @45°: 96.2 -> 96.6 | Helped |
| Set fusion weight ω to 0.25 instead of 0.50 (ablation) | Generative GeoTrans · 3DMatch | Rotation accuracy @45°: 96.2 -> 97.2 | Helped |
| Set fusion weight ω to 0.75 instead of 0.50 (ablation) | Generative GeoTrans · 3DMatch | Rotation accuracy @45°: 96.2 -> 93.9 | Hurt |
Surprising findings
- Generative ColorPCR with synthesized RGB outperforms ColorPCR with real RGB on 3DMatch, which the authors attribute to fewer calibration and lighting errors in real RGB-D data.
- Stable Diffusion features give higher precision than DINOv2 features for the zero-shot color fusion.
Limitations
- Zero-shot generated pairs can still show geometric or texture inconsistencies in some regions, requiring few-shot fine-tuning.
- Evaluated only on indoor RGB-D benchmarks (3DMatch, ScanNet).
Experimental setup
Datasets: ScanNet 3DMatch
Benchmarks: 3DMatch ScanNet
Metrics: rotation accuracy translation accuracy Chamfer accuracy mean error median error
Baselines: FPFH Lepard RegTR RoITr FCGF Predator GeoTransformer ColorPCR
Builds on: ControlNet Stable Diffusion DINOv2 ColorPCR
Closest papers by meaning
- Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment (AAAI 2025 · similarity 0.89)
- Zero-shot RGB-D Point Cloud Registration with Pre-trained Large Vision Model (CVPR 2025 · similarity 0.88)
- Partial Point Cloud Registration with Multi-view 2D Image Learning (AAAI 2025 · similarity 0.88)
- Cross-PCR: A Robust Cross-Source Point Cloud Registration Framework (AAAI 2025 · similarity 0.87)
- FreeReg: Image-to-Point Cloud Registration Leveraging Pretrained Diffusion Models and Monocular Depth Estimators (ICLR 2024 · similarity 0.87)
- Where Precision Meets Efficiency: Transformation Diffusion Model for Point Cloud Registration (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.