Research map / Point clouds / Shape and surface completion
LAM3D: Large Image-Point Clouds Alignment Model for 3D Reconstruction from Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
LAM3D compresses point clouds into latent tri-planes with a point-cloud autoencoder, then trains three parallel diffusion UNets to align single-image DINO features to those tri-planes for high-fidelity single-image mesh reconstruction in about 6 seconds.
Problem
Single-image 3D reconstruction with large reconstruction models produces geometric distortions because 3D shape is ambiguous from one image and multi-view images from diffusion are inconsistent.
Approach
Use point-cloud-derived latent tri-planes (with SDF supervision on latent and reconstructed planes) as a 3D-aware target, and align image features to them with probabilistic diffusion using three parallel UNets, one per plane, rather than deterministic transformers or contrastive alignment like ULIP.
Key results
- On Google Scanned Objects, LAM3D reaches Chamfer Distance 0.0083, Volume IoU 0.6235 and F-Score 85.40%, beating the best baseline CRM (0.0094, 0.6131, 79.38%).
- Mesh generation takes about 6 seconds on one NVIDIA V100 GPU.
- On the ablation set, the latent tri-plane with SDF loss reaches CD 1.81e-4, versus 4.81e-4 for a flat vector latent and 35.23e-4 for ULIP-based alignment; parallel UNets reduce CD from 3.92e-3 to 2.59e-3 compared with a single UNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use LAM3D instead of CRM | Single-image 3D reconstruction · Google Scanned Objects | Chamfer Distance: 0.0094 -> 0.0083 | Helped |
| use LAM3D instead of CRM | Single-image 3D reconstruction · Google Scanned Objects | Volume IoU: 0.6131 -> 0.6235 | Helped |
| use LAM3D instead of CRM | Single-image 3D reconstruction · Google Scanned Objects | F-Score: 79.38 -> 85.40 | Helped |
| replace a vector latent with a spatial latent tri-plane (ablation) | LAM3D point-cloud compressor · Objaverse 216-object evaluation | CD (× 10^4): 4.81 -> 1.95 | Helped |
| add latent SDF loss (ablation) | LAM3D point-cloud compressor · Objaverse 216-object evaluation | CD (× 10^4): 1.95 -> 1.81 | Helped |
| use parallel diffusion UNets rather than a single UNet (ablation) | LAM3D image-point-cloud alignment · Objaverse 216-object evaluation | CD (× 10^3): 3.92 -> 2.59 | Helped |
| increase spatial resolution from 8×8 to 16×16 (feature dim 2) (ablation) | LAM3D point-cloud compressor · Point cloud to 3D mesh | CD (× 10^4): 4.15 -> 2.12 | Helped |
| increase spatial resolution from 16×16 to 32×32 (feature dim 2) (ablation) | LAM3D point-cloud compressor · Point cloud to 3D mesh | CD (× 10^4): 2.12 -> 1.81 | Helped |
| increase feature dimension from 1 to 2 at 32×32 resolution (ablation) | LAM3D point-cloud compressor · Point cloud to 3D mesh | CD (× 10^4): 1.87 -> 1.81 | Helped |
| increase feature dimension from 2 to 4 at 32×32 resolution (ablation) | LAM3D point-cloud compressor · Point cloud to 3D mesh | CD (× 10^4): 1.81 -> 2.63 | Hurt |
| add the plane refiner (ablation) | LAM3D point-cloud compressor · Point cloud to 3D mesh | CD (× 10^4): 2.37 -> 1.81 | Helped |
| replace ULIP vector feature alignment with LAM3D alignment | Image-to-3D reconstruction · Objaverse 216-object evaluation | CD (× 10^4): 35.23 -> 1.81 | Helped |
Surprising findings
- A deterministic transformer alignment was unstable, so the authors switched to diffusion for probabilistic alignment.
- Replacing the aligned image features in ULIP with a decoder gave much worse Chamfer distance (35.23e-4) than the proposed method, suggesting contrastive alignment is poorly suited to reconstruction.
Limitations
- Only geometry is reconstructed; textured mesh generation is not supported.
- Training is expensive: 32 V100 GPUs for about 3 days in stage 1 and 8 V100 GPUs for 2 days in stage 2.
- Increasing latent tri-plane resolution beyond 32x32 becomes computationally intractable.
Experimental setup
Datasets: Objaverse Google Scanned Objects
Benchmarks: Google Scanned Objects Objaverse (unseen objects)
Metrics: Chamfer Distance Volume IoU F-Score
Baselines: One-2-3-45 LRM / OpenLRM SyncDreamer Wonder3D Magic123 TGS LGM CRM ULIP
Builds on: Large Reconstruction Model (LRM) ULIP DDPM Marching Cubes DINO
Closest papers by meaning
- EdgeMovingNet: Edge-preserving Point Cloud Reconstruction via Joint Geometry Features (CVPR 2025 · similarity 0.88)
- SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images (CVPR 2025 · similarity 0.87)
- Hyperbolic-Constraint Point Cloud Reconstruction from Single RGB-D Images (AAAI 2025 · similarity 0.86)
- UniPre3D: Unified Pre-training of 3D Point Cloud Models with Cross-Modal Gaussian Splatting (CVPR 2025 · similarity 0.86)
- X-3D: Explicit 3D Structure Modeling for Point Cloud Recognition (CVPR 2024 · similarity 0.85)
- Bridge 2D-3D: Uncertainty-aware Hierarchical Registration Network with Domain Alignment (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.