Research map / Point clouds / Shape and surface completion
Neural Point Cloud Diffusion for Disentangled 3D Shape and Appearance Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Point Cloud Diffusion (NPCD), a denoising diffusion model over category-level Point-NeRF neural point clouds (positions plus high-dimensional features), enabling independent sampling and control of 3D shape and appearance.
Problem
Existing 3D diffusion models generate shape and appearance jointly, so they cannot control coarse shape and local appearance separately; prior disentangled generators (GRAF, Disentangled3D) are GAN-based with weaker quality.
Approach
Represent objects as neural point clouds (point positions for coarse shape, high-dimensional point features for local appearance and geometry rendered by a Point-NeRF-style renderer), train a category-level autodecoder with zero initialization and TV/KL regularization to avoid many-to-one feature mappings, then jointly diffuse positions and features with a transformer denoiser; disentangled sampli
Key results
- Disentangled generation FID on SRN cars 28.38 vs GRAF 40.95 and D3D 62.34; PhotoShape chairs FID 14.45 vs GRAF 34.49.
- On PhotoShape chairs NPCD FID 14.45 beats DiffRF 15.95; on SRN cars SSDNeRF (11.08 FID) remains better than NPCD (28.38).
- Zero initialization plus TV+KL regularization raises per-point feature cosine similarity from 0.0306 (random init) to about 0.95.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NPCD instead of GRAF for disentangled generation | 3D shape and appearance generation · ShapeNet SRN Cars | FID: 40.95 -> 28.38 | Helped |
| Use NPCD instead of GRAF for disentangled generation | 3D shape and appearance generation · ShapeNet SRN Chairs | FID: 37.19 -> 9.87 | Helped |
| Use NPCD instead of GRAF for disentangled generation | 3D shape and appearance generation · PhotoShape Chairs | FID: 34.49 -> 14.45 | Helped |
| Use NPCD instead of DiffRF | 3D shape and appearance generation · PhotoShape Chairs | FID: 15.95 -> 14.45 | Helped |
| Use NPCD instead of SSDNeRF | 3D shape and appearance generation · SRN Cars | FID: 11.08 -> 28.38 | Hurt |
| Use NPCD instead of LION for point-cloud generation | NPCD · SRN Cars | CD: 53.41 -> 60.23 | Hurt |
| Use NPCD instead of LION for point-cloud generation | NPCD · SRN Cars | EMD: 51.14 -> 52.41 | Hurt |
| Use NPCD instead of LION for point-cloud generation | NPCD · SRN Chairs | CD: 53.70 -> 60.50 | Hurt |
| Use NPCD instead of LION for point-cloud generation | NPCD · SRN Chairs | EMD: 52.34 -> 58.84 | Hurt |
| Replace random feature initialization with zero initialization (ablation) | Point-NeRF autodecoder · Neural point feature space | Cosine sim.: 0.0306 -> 0.7695 | Helped |
| Add TV regularization to zero-initialized features (ablation) | Point-NeRF autodecoder · Neural point feature space | Cosine sim.: 0.7695 -> 0.9355 | Helped |
| Add KL regularization to zero-initialized features (ablation) | Point-NeRF autodecoder · Neural point feature space | Cosine sim.: 0.7695 -> 0.9480 | Helped |
Surprising findings
- SSDNeRF outperforms NPCD on SRN cars FID (11.08 vs 28.38), so the method is not uniformly state of the art among unconditional 3D diffusion models.
Limitations
- Trained only on object categories with per-object point clouds extracted from meshes; generalization beyond ShapeNet/PhotoShape chairs and cars is not shown.
- Worse FID than SSDNeRF on SRN cars.
- Point positions are assumed given for training objects, so the pipeline depends on mesh-derived point clouds.
Experimental setup
Datasets: ShapeNet SRN cars ShapeNet SRN chairs PhotoShape chairs
Benchmarks: ShapeNet SRN cars ShapeNet SRN chairs PhotoShape chairs
Metrics: FID KID 1-NN accuracy with Chamfer distance 1-NN accuracy with EMD mean cosine similarity of features
Baselines: GRAF Disentangled3D Functa SSDNeRF DiffRF PVD LION Point-E
Builds on: Point-NeRF DDPM RePaint Disentangled3D NeRF
Closest papers by meaning
- SIGNeRF: Scene Integrated Generation for Neural Radiance Fields (CVPR 2024 · similarity 0.88)
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.88)
- DreamCraft3D: Hierarchical 3D Generation with Bootstrapped Diffusion Prior (ICLR 2024 · similarity 0.87)
- Denoising Diffusion via Image-Based Rendering (ICLR 2024 · similarity 0.87)
- Generating Images with 3D Annotations Using Diffusion Models (ICLR 2024 · similarity 0.87)
- GaussianAnything: Interactive Point Cloud Flow Matching for 3D Generation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.