Research map / 3D vision / 3D generation
High-fidelity 3D Object Generation from Single Image with RGBN-Volume Gaussian Reconstruction Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GS-RGBN, a feed-forward single-image-to-3D reconstruction model that builds a hybrid voxel-Gaussian representation from Wonder3D RGB and normal views and fuses them with a cross-volume fusion module to regress 2D Gaussians.
Problem
Single-view 3D generation via Gaussian splatting from multi-view diffusion outputs suffers from view inconsistency and unstructured 3D Gaussians, producing distorted and blurry geometry.
Approach
Constrain each Gaussian to a voxel grid of back-projected DINO RGB and normal features so 3D convolutions can exploit neighborhood structure, and fuse RGB and normal volumes with cross-attention to recover geometric detail; decode with 2D Gaussian splatting and depth/normal regularization.
Key results
- On GSO novel view synthesis, GS-RGBN reaches 23.02 PSNR, 0.873 SSIM, 0.135 LPIPS, versus 17.43 PSNR for the best baseline DreamGaussian (and 17.13 for LGM).
- Chamfer Distance of 27.49 (x10^-3) vs 58.74 for TriplaneGaussian, the next best.
- Ablation: removing the normal input drops PSNR to 20.15 and removing CVF drops it to 19.27 from the full 23.02.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GS-RGBN instead of TriplaneGaussian | single-view 3D reconstruction · GSO | PSNR: 16.73 -> 23.02 | Helped |
| use GS-RGBN instead of LGM | single-view 3D reconstruction · GSO | SSIM: 0.808 -> 0.873 | Helped |
| use GS-RGBN instead of One-2-3-45 | single-view 3D reconstruction · GSO | LPIPS: 0.231 -> 0.135 | Helped |
| use GS-RGBN instead of TriplaneGaussian | single-view 3D reconstruction · GSO | CD (×10^-3): 58.74 -> 27.49 | Helped |
| replace the hybrid Voxel-Gaussian representation with Image-Gaussian (ablation) | GS-RGBN · GSO | PSNR: 23.02 -> 18.82 | Hurt |
| add LPIPS loss (ablation) | GS-RGBN · GSO | PSNR: 21.83 -> 23.02 | Helped |
| add depth loss (ablation) | GS-RGBN · GSO | PSNR: 21.62 -> 23.02 | Helped |
| add regularization loss (ablation) | GS-RGBN · GSO | PSNR: 22.51 -> 23.02 | Helped |
| add multi-view normal-map input (ablation) | GS-RGBN · GSO | PSNR: 20.15 -> 23.02 | Helped |
| add CVF module (ablation) | GS-RGBN · GSO | PSNR: 19.27 -> 23.02 | Helped |
| add RGB-guided cross-attention (CAs) (ablation) | GS-RGBN · GSO | PSNR: 21.08 -> 23.02 | Helped |
| add normal-guided cross-attention (CAg) (ablation) | GS-RGBN · GSO | PSNR: 21.32 -> 23.02 | Helped |
Limitations
- Performance degrades when the multi-view diffusion model produces highly inconsistent views.
- Voxel representation does not scale directly to large-scale scenes; octrees are suggested as future fix.
Experimental setup
Datasets: Objaverse-LVIS Google Scanned Objects
Benchmarks: Google Scanned Objects
Metrics: PSNR SSIM LPIPS Chamfer Distance runtime
Baselines: DreamGaussian LGM One-2-3-45 Wonder3D TriplaneGaussian
Builds on: Large Reconstruction Model (LRM) 2D Gaussian Splatting Wonder3D LGM TriplaneGaussian
Closest papers by meaning
- GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling (NeurIPS 2024 · similarity 0.89)
- Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.88)
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.88)
- Text-to-3D using Gaussian Splatting (CVPR 2024 · similarity 0.88)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.88)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.