Research map / 3D vision / 3D generation
LRM: Large Reconstruction Model for Single Image to 3D
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LRM, a 500M-parameter transformer encoder-decoder that regresses a triplane NeRF from a single image in about 5 seconds, trained end-to-end on ~1M Objaverse and MVImgNet objects with photometric reconstruction losses.
Problem
Single-image 3D reconstruction has been limited to small datasets and category-specific models, or requires slow per-shape optimization or diffusion-based multi-view generation; the paper asks whether a large generic 3D prior can be learned from a single image.
Approach
A large transformer decoder maps DINO image tokens to triplane tokens via cross-attention, conditioned on normalized camera features with adaLN modulation; the triplane is decoded to a NeRF and trained end-to-end by rendering novel views, with no 2D diffusion or 3D-aware regularization.
Key results
- On 100 GSO objects, LRM reaches FID 31.44 and PSNR 19.60 versus 97.05 FID and 14.36 PSNR for Shap-E.
- LRM achieves Chamfer Distance 0.053 versus 0.085 for Shap-E on GSO.
- Reconstruction takes about 5 seconds per shape on a single NVIDIA A100 GPU, including mesh extraction.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LRM instead of Shap-E | Single-image 3D reconstruction · Google Scanned Objects | FID: 97.05 -> 31.44 | Helped |
| Use LRM instead of Shap-E | Single-image 3D reconstruction · Google Scanned Objects | CLIP-Similarity: 0.805 -> 0.902 | Helped |
| Combine synthetic Objaverse and real MvImgNet training data | LRM analysis baseline · Unseen Evaluation | PSNR: 15.5 -> 19.0 | Helped |
| Add real MvImgNet data to synthetic Objaverse training (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 15.5 -> 19.0 | Helped |
| Increase training views per shape from 4 to 16 (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 18.8 -> 19.1 | Helped |
| Increase decoder cross-attention layers from 16 to 24 (ablation) | LRM analysis baseline · Unseen Evaluation | LPIPS: 19.1 -> 18.9 | Helped |
| Increase decoder cross-attention layers from 16 to 24 (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 19.0 -> 19.1 | Helped |
| Reduce NeRF MLP layers from 12 to 2 (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 19.0 -> 19.2 | Helped |
| Increase triplane resolution from 64up to 128up (ablation) | LRM analysis baseline · Unseen Evaluation | CLIP-Similarity: 87.8 -> 88.3 | Helped |
| Normalize camera poses instead of leaving them unmodified (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 15.3 -> 19.0 | Helped |
| Normalize camera poses instead of using random camera rotations (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 18.0 -> 19.0 | Helped |
| Increase supervised side views per training sample from 3 to 4 (ablation) | LRM analysis baseline · Unseen Evaluation | PSNR: 19.0 -> 19.1 | Helped |
Surprising findings
- Training on real MVImgNet video data alone outperforms training on synthetic Objaverse alone, and combining them is substantially better than either.
- The authors note LRM does not show an advantage over Point-E and Shap-E in parameter count or dataset size, attributing its gains to the triplane representation and end-to-end training.
Limitations
- Produces blurry textures for occluded regions because the deterministic model averages plausible modes.
- Assumes fixed camera intrinsics and extrinsics at test time, which can distort shapes when test images are cropped or resized.
- Handles only background-free single objects and assumes Lambertian appearance, so view-dependent materials are not reproduced.
Experimental setup
Datasets: Objaverse MVImgNet Google Scanned Objects ImageNet Amazon Berkeley Objects
Benchmarks: Google Scanned Objects Objaverse unseen shapes MVImgNet unseen videos
Metrics: FID CLIP-Similarity PSNR LPIPS Chamfer Distance SSIM
Baselines: Point-E Shap-E One-2-3-45 MCC GINA-3D Zero-1-to-3
Builds on: NeRF EG3D triplane DINO DiT adaLN Perceiver
Closest papers by meaning
- LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields (CVPR 2025 · similarity 0.91)
- LRM-Zero: Training Large Reconstruction Models with Synthesized Data (NeurIPS 2024 · similarity 0.91)
- PF-LRM: Pose-Free Large Reconstruction Model for Joint Pose and Shape Prediction (ICLR 2024 · similarity 0.89)
- GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement (ICLR 2025 · similarity 0.89)
- 4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time (NeurIPS 2025 · similarity 0.88)
- MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.