Research map / 3D vision / 3D generation
GeoLRM: Geometry-Aware Large Reconstruction Model for High-Quality 3D Gaussian Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GeoLRM, a feed-forward large reconstruction model that predicts 3D Gaussians from many posed views using a two-stage 3D-anchor pipeline and deformable cross-attention, scaling to 21 input views.
Problem
Existing large reconstruction models use triplanes or pixel-aligned Gaussians that waste capacity on empty space and ignore 3D-2D projection geometry, so they do not improve (and often degrade) when given denser input views, limiting resolution and quality for image-to-3D generation.
Approach
Operate directly on sparse 3D anchor points (occupied voxels predicted by a proposal transformer) and fuse multi-view image features via deformable cross-attention that projects each 3D token onto each input view, with 3D RoPE for self-attention, so the cost scales with occupied voxels rather than dense views.
Key results
- On GSO with 6 input views, GeoLRM reaches 23.57 PSNR / 0.872 SSIM / 0.167 CD, versus InstantMesh's 23.19 / 0.856 / 0.186, using 4.92 GB memory versus 23.12 GB.
- On GSO, GeoLRM's PSNR rises from 22.84 (4 views) to 25.13 (20 views), whereas InstantMesh stays around 23 and degrades at 12 views.
- On OmniObject3D, GeoLRM achieves 24.74 PSNR, 0.883 SSIM, 0.134 LPIPS, outperforming InstantMesh (23.86 PSNR).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Plücker ray embeddings (ablation) | GeoLRM image encoder · GSO | PSNR: 20.64 -> 20.73 | Helped |
| add low-level features (ablation) | GeoLRM image encoder · GSO | PSNR: 20.29 -> 20.73 | Helped |
| add high-level features (ablation) | GeoLRM image encoder · GSO | PSNR: 15.85 -> 20.73 | Helped |
| add 3D RoPE (ablation) | GeoLRM reconstruction transformer · GSO | PSNR: 20.52 -> 20.73 | Helped |
| use dynamic rather than fixed input-view counts (ablation) | GeoLRM · GSO, 6 input views | PSNR: 20.97 -> 20.73 | Hurt |
| increase deformable-attention sampling points from 0 to 4 (ablation) | GeoLRM reconstruction transformer · GSO | PSNR: 19.52 -> 20.21 | Helped |
| increase deformable-attention sampling points from 4 to 8 (ablation) | GeoLRM reconstruction transformer · GSO | PSNR: 20.21 -> 20.73 | Helped |
| increase deformable-attention sampling points from 8 to 16 (ablation) | GeoLRM reconstruction transformer · GSO | PSNR: 20.73 -> 20.80 | Helped |
| increase input views from 4 to 20 | GeoLRM · GSO | PSNR: 22.84 -> 25.13 | Helped |
Surprising findings
- InstantMesh's quality does not improve and even declines when input views increase beyond 8, contrary to the expectation that more views help.
- Dynamic input-view training slightly lowers performance under matched fixed-view testing (20.97 vs 20.73 PSNR for fixed vs full model in the ablation table), though it improves generalization across input configurations.
Limitations
- Two-stage pipeline is not end-to-end, so errors from the proposal network accumulate.
- Reliance on the proposal network is needed because processing Gaussians over the full 3D space is computationally intensive.
- Mixed-precision training fails for the reconstruction transformer stage, which is sensitive to data type.
Experimental setup
Datasets: GObjaverse Objaverse Google Scanned Objects OmniObject3D
Benchmarks: Google Scanned Objects OmniObject3D
Metrics: PSNR SSIM LPIPS Chamfer Distance F-Score inference time memory
Baselines: LGM CRM InstantMesh TripoSR OpenLRM
Builds on: LRM Deformable DETR deformable attention DINOv2 RoPE 3D Gaussian splatting
Closest papers by meaning
- GTR: Improving Large 3D Reconstruction Models through Geometry and Texture Refinement (ICLR 2025 · similarity 0.87)
- G3DR: Generative 3D Reconstruction in ImageNet (CVPR 2024 · similarity 0.87)
- CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction (NeurIPS 2026 · similarity 0.86)
- Large Spatial Model: End-to-end Unposed Images to Semantic 3D (NeurIPS 2024 · similarity 0.86)
- Triplane Meets Gaussian Splatting: Fast and Generalizable Single-View 3D Reconstruction with Transformers (CVPR 2024 · similarity 0.86)
- SGCR: Spherical Gaussians for Efficient 3D Curve Reconstruction (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.