Research map / Point clouds / Shape and surface completion
PCDreamer: Point Cloud Completion Through Multi-view Diffusion Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PCDreamer, which uses multi-view diffusion priors (ControlNet + Wonder3D/Stable Video Diffusion + DepthAnything) to generate depth views of a partial point cloud, then fuses them with the input via attention and filters noisy points to produce complete point clouds.
Problem
Point cloud completion from single-view partial point clouds, where large missing regions create a vast solution space and paired image-point cloud data is hard to obtain.
Approach
Use off-the-shelf multi-view diffusion models to 'dream' view-consistent depth images of the missing shape from a single-view depth render, then fuse these images with the partial point cloud via attention and apply confidence-based point filtering and offset-based upsampling to get the final completion.
Key results
- On PCN, Ours(Wonder3D) reaches CD-Avg 6.49 and F1 0.859 versus 6.56 and 0.846 for the best prior method CRA-PCN.
- On ShapeNet-55 (10 shown classes), Ours(Wonder3D) achieves CD-Avg 0.895 versus 0.938 for SVDFormer.
- Depth-only multi-view signal outperforms RGB-only and RGB+depth combinations on PCN (CD-Avg 6.49 vs 6.63 and 6.59).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse partial point cloud features with multi-view depth-image features instead of using depth images alone (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 17.69 -> 6.49 | Helped |
| Fuse partial point cloud features with multi-view depth-image features instead of using partial point cloud alone (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.91 -> 6.49 | Helped |
| Use depth images rather than RGB images as the multi-view signal (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.63 -> 6.49 | Helped |
| Use depth images rather than combined RGB and depth images as the multi-view signal (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.59 -> 6.49 | Helped |
| Add confidence-based point filtering (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.76 -> 6.49 | Helped |
| Use 6 views instead of 2 views (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.65 -> 6.52 | Helped |
| Use 6 views instead of 4 views (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.64 -> 6.52 | Helped |
| Use 6 views instead of 8 views (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.62 -> 6.52 | Helped |
| Filter 25% rather than 12% of points (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.92 -> 6.49 | Helped |
| Filter 50% rather than 25% of points (ablation) | PCDreamer · PCN test | CD-Avg (ℓ1 ×103): 6.49 -> 6.47 | No clear effect |
Surprising findings
- RGB images generated by ControlNet introduce artifacts that hurt completion, so depth-only multi-view inputs work best.
- Image-only completion and point-cloud-only completion are both weaker than the fused model, and a direct 3D fusion of multi-view depths was found to be noisy.
Limitations
- Slender structures (e.g., earphones) yield inconsistent multi-view depth images, causing completion failures.
- Relies on a fixed viewpoint and a pipeline of several large pretrained diffusion models, which adds cost.
- Experiments are on synthetic ShapeNet-derived data; real-world LiDAR scans are only discussed in the supplementary.
Experimental setup
Datasets: PCN ShapeNet-55 ShapeNet-34
Benchmarks: PCN ShapeNet-55 ShapeNet-34
Metrics: Chamfer distance density-aware Chamfer distance F1-Score@1%
Baselines: PCN GRNet PoinTr PMP-Net++ SnowFlakeNet SeedFormer AnchorFormer HyperCD SVDFormer CRA-PCN
Builds on: Wonder3D Stable Video Diffusion ControlNet DepthAnything PoinTr
Closest papers by meaning
- DiffCorr: Conditional Diffusion Model with Reliable Pseudo-Label Guidance for Unsupervised Point Cloud Shape Correspondence (AAAI 2025 · similarity 0.90)
- Digging into Intrinsic Contextual Information for High-fidelity 3D Point Cloud Completion (AAAI 2025 · similarity 0.89)
- CDPNet: Cross-Modal Dual Phases Network for Point Cloud Completion (AAAI 2024 · similarity 0.88)
- ComPC: Completing a 3D Point Cloud with 2D Diffusion Priors (ICLR 2025 · similarity 0.88)
- Position-Aware Guided Point Cloud Completion with CLIP Model (AAAI 2025 · similarity 0.88)
- CRA-PCN: Point Cloud Completion with Intra- and Inter-level Cross-Resolution Transformers (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.