Research map / 3D vision / 3D generation
Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Forward Curvature-Matching (FCM), an adjoint-free adaptive step-size likelihood update for DDIM-based point cloud diffusion sampling, enabling image- and depth-conditioned 3D reconstruction without retraining.
Problem
Image-to-point-cloud reconstruction with diffusion models: posterior-learning methods need conditioning at training and fixed view counts, while prior+likelihood (DPS-style) methods use heuristic fixed step sizes that converge slowly, and rendering operators lack tractable adjoints.
Approach
Choose the likelihood-gradient step size per iteration from a forward-only finite-difference curvature estimate (Barzilai-Borwein style, capped at 1/L, with a single Armijo halving), so no renderer adjoint is needed and the diffusion prior is trained once and reused across measurement types and view counts.
Key results
- Single-view ShapeNet average F-score 0.382 vs 0.343 (BDM) and 0.310 (PC$^2$) at matched NFEs; EMD 0.587 vs 0.647/0.648.
- Average CD on ShapeNet is 0.561 for ours vs 0.574 (PC$^2$) and 0.588 (BDM), i.e. best among the three in the main table.
- Ours uses 256 NFEs versus 1000 for PC$^2$ and 1080 for BDM in the runtime comparison, with a reported 32.1% runtime reduction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FCM-guided diffusion reconstruction instead of PC² | ShapeNet single-view reconstruction · ShapeNet single-view average | F-score: 0.310 -> 0.382 | Helped |
| Use FCM-guided diffusion reconstruction instead of BDM | ShapeNet single-view reconstruction · ShapeNet single-view average | EMD (×10): 0.648 -> 0.587 | Helped |
| Set L to 100 rather than the full method's L=2/3 (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.307 | Hurt |
| Set L to 10 rather than the full method's L=2/3 (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.373 | Hurt |
| Set L to 1 rather than the full method's L=2/3 (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.376 | Hurt |
| Set η_FCM to 10⁻⁶ rather than the full method's 10⁻⁴ (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.373 | Hurt |
| Set η_FCM to 10⁻⁵ rather than the full method's 10⁻⁴ (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.377 | Hurt |
| Set δ₀ to 5×10⁻² rather than the full method's 2×10⁻² (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.343 | Hurt |
| Set δ₀ to 10⁻² rather than the full method's 2×10⁻² (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.369 | Hurt |
| Set δ₀ to 10⁻³ rather than the full method's 2×10⁻² (ablation) | FCM-guided sampling · ShapeNet FCM hyperparameter study | F-score: 0.382 -> 0.330 | Hurt |
Surprising findings
- The paper reports that under the ShapeNet comparison, CD is worse than the baselines on several categories (e.g., chair and table), and the authors attribute this to direct point-position control during likelihood updates.
Limitations
- Reconstructed point clouds appear thinner than ground truth due to rasterizer radius; thin structures affected.
- ShapeNet colors come from KeypointNet part-based coloring rather than true mesh colors, which may degrade quality.
- Direct likelihood updates on point positions can limit CD performance relative to prior-only methods.
Experimental setup
Datasets: KeypointNet ShapeNet rendered images CO3D
Benchmarks: ShapeNet (airplane, car, chair, table) CO3D (hydrant, teddybear)
Metrics: EMD L1 Chamfer Distance F-score@0.01
Baselines: PC$^2$ BDM DDPM+DPS DDIM+DPS
Builds on: Diffusion Posterior Sampling DDIM DDPM Barzilai-Borwein method Armijo line search
Closest papers by meaning
- SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images (CVPR 2025 · similarity 0.90)
- Coherent 3D Scene Diffusion From a Single RGB Image (NeurIPS 2024 · similarity 0.89)
- Decompositional Neural Scene Reconstruction with Generative Diffusion Prior (CVPR 2025 · similarity 0.88)
- Multi-view Reconstruction via SfM-guided Monocular Depth Estimation (CVPR 2025 · similarity 0.88)
- Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking (CVPR 2024 · similarity 0.88)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.