Research map / 3D vision / 3D generation
LRM-Zero: Training Large Reconstruction Models with Synthesized Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a GS-LRM-style sparse-view reconstruction transformer (LRM-Zero) entirely on procedurally synthesized primitive-based 3D data (Zeroverse), achieving results competitive with an Objaverse-trained model.
Problem
3D data for training large reconstruction models is scarce, expensive to create, and raises licensing, bias, and identity concerns; it is unclear whether semantic real-world 3D data is necessary for sparse-view reconstruction.
Approach
Procedurally compose random textured primitives (cube, sphere, cylinder, cone, torus) and apply height-field, boolean-difference, and wireframe augmentations to create a non-semantic synthetic dataset, then train a standard transformer LRM on it; reconstruction is argued to rely mainly on local cues rather than global semantics.
Key results
- On 8-view GSO at Res-256, LRM-Zero reaches 30.78 PSNR / 0.957 SSIM / 0.036 LPIPS vs GS-LRM's 31.90 / 0.966 / 0.030 (gap about 1.1 PSNR).
- On 8-view ABO at Res-256, LRM-Zero gets 28.82 PSNR vs GS-LRM's 30.66.
- On OpenIllumination, LRM-Zero (14.44 PSNR) slightly outperforms GS-LRM (14.02), while on OmniObject3D GS-LRM is better (29.32 vs 25.81).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train GS-LRM on Zeroverse synthetic data instead of Objaverse | GS-LRM · GSO, 8 views, Res-256 | PSNR: 31.90 -> 30.78 | Hurt |
| Train GS-LRM on Zeroverse synthetic data instead of Objaverse | GS-LRM · ABO, 8 views, Res-256 | PSNR: 30.66 -> 28.82 | Hurt |
| Add boolean-difference augmentation (ablation) | LRM-Zero · GSO | PSNR: 29.88 -> 30.75 | Helped |
| Add boolean-difference augmentation (ablation) | LRM-Zero · ABO | PSNR: 27.39 -> 28.76 | Helped |
| Add wireframe augmentation (ablation) | LRM-Zero · GSO | PSNR: 30.75 -> 30.78 | No clear effect |
| Add wireframe augmentation (ablation) | LRM-Zero · ABO | PSNR: 28.76 -> 28.82 | No clear effect |
| Add height-field augmentation (ablation) | LRM-Zero · GSO | PSNR: 29.22 -> 30.24 | Helped |
| Add height-field augmentation (ablation) | LRM-Zero · ABO | PSNR: 27.70 -> 28.31 | Helped |
| Increase training steps from 1x to 2x (ablation) | LRM-Zero · GSO | PSNR: 30.78 -> 31.47 | Helped |
| Increase training steps from 1x to 2x (ablation) | LRM-Zero · ABO | PSNR: 28.82 -> 29.33 | Helped |
| Increase model size from 1x to 2x at 1x training steps (ablation) | LRM-Zero · GSO | PSNR: 31.47 -> 30.19 | Hurt |
| Increase data size from 1x to 4x at 2x training steps (ablation) | LRM-Zero · GSO | PSNR: 31.47 -> 31.15 | Hurt |
Surprising findings
- Training on 200K instead of 800K Objaverse objects drops GS-LRM only 0.1 PSNR on GSO, suggesting saturation for single-object reconstruction with realistic data.
- Training stability was highly sensitive to data complexity; boolean-difference-heavy mixtures caused gradient explosion and divergence, requiring a model-data co-design.
Limitations
- Gap to Objaverse-trained GS-LRM persists, especially with invisible regions under sparse views, attributed to lack of semantics for hallucination.
- Zeroverse lacks semantics, so it is unsuitable for tasks like single-view reconstruction.
- Scaling to more data and larger models gave inconsistent gains; training convergence and hyperparameters needed retuning, and larger models were not fully explored.
Experimental setup
Datasets: Zeroverse Objaverse GSO ABO OpenIllumination OmniObject3D
Benchmarks: GSO ABO OpenIllumination OmniObject3D Objaverse-test Zeroverse-test
Metrics: PSNR SSIM LPIPS
Baselines: GS-LRM trained on Objaverse NeRF-LRM trained on Objaverse
Builds on: GS-LRM LRM Objaverse Procedural material/relighting synthetic pipeline (Xu et al.)
Closest papers by meaning
- LRM: Large Reconstruction Model for Single Image to 3D (ICLR 2024 · similarity 0.91)
- MegaSynth: Scaling Up 3D Scene Reconstruction with Synthesized Data (CVPR 2025 · similarity 0.89)
- LIRM: Large Inverse Rendering Model for Progressive Reconstruction of Shape, Materials and View-dependent Radiance Fields (CVPR 2025 · similarity 0.89)
- ZeroShape: Regression-based Zero-shot Shape Reconstruction (CVPR 2024 · similarity 0.88)
- L4GM: Large 4D Gaussian Reconstruction Model (NeurIPS 2024 · similarity 0.88)
- 4D-LRM: Large Space-Time Reconstruction Model From and To Any View at Any Time (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.