Research map / 3D vision / 3D generation
GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GaussianCube, a structured, fully explicit radiance representation built from a fixed number of 3D Gaussians (densification-constrained fitting) rearranged into a voxel grid via optimal transport, enabling a standard 3D U-Net diffusion model for 3D generation.
Problem
Existing radiance representations for 3D generative modeling either rely on a shared implicit feature decoder (degrading representation power) or are spatially unstructured (3D Gaussians), making them hard to use with standard 3D diffusion architectures.
Approach
Fit each object with a fixed number of Gaussians by constraining densification (not pruning) to a maximum N^3, then map Gaussians to voxel centers via an optimal transport (Jonker-Volgenant) assignment, storing offsets from voxel centers so the result is a dense 3D grid that a standard 3D U-Net diffusion model can denoise.
Key results
- On ShapeNet Car fitting, GaussianCube reaches 34.94 PSNR with 0.46M parameters versus 31.78 PSNR and 67.12M parameters for voxels and 32.61 PSNR and 6.30M for triplane.
- Unconditional ShapeNet Car generation achieves FID-50K 13.01 vs 17.15 for GET3D and 30.48 for EG3D.
- Text-to-3D on Objaverse achieves CLIP score 30.56 with 2.30 s inference, versus 30.52 for Shap-E and 26.38 for DreamGaussian.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GaussianCube instead of GET3D | 3D generation · ShapeNet Car | FID-50K: 17.15 -> 13.01 | Helped |
| use GaussianCube instead of GET3D | 3D generation · ShapeNet Chair | FID-50K: 19.24 -> 15.99 | Helped |
| use GaussianCube instead of DiffTF | class-conditioned 3D generation · OmniObject3D | FID-50K: 46.06 -> 11.62 | Helped |
| use GaussianCube instead of Rodin | image-conditioned avatar generation · Synthetic Avatar | CSIM: 0.6466 -> 0.7821 | Helped |
| use GaussianCube instead of Shap-E | text-to-3D generation · Objaverse | CLIP Score: 30.52 -> 30.56 | Helped |
| use GaussianCube instead of LGM | text-to-3D generation · Objaverse | Inference Time (s): 1.55 -> 2.30 | Hurt |
| remove Optimal Transport organization (ablation) | GaussianCube generation · ShapeNet Car | FID-50K: 13.01 -> 21.41 | Hurt |
| omit densification and pruning and anchor Gaussians to the voxel grid without offsets (ablation) | GaussianCube fitting · ShapeNet Car | PSNR: 34.94 -> 25.87 | Hurt |
| add learnable offsets to the voxel grid while omitting densification and pruning (ablation) | GaussianCube fitting · ShapeNet Car | PSNR: 34.94 -> 30.18 | Hurt |
| increase Nmax from 4096 to 262144 (ablation) | densification-constrained GaussianCube fitting · ShapeNet Car | PSNR: 32.56 -> 35.34 | Helped |
| use classifier-free guidance at scale 2.0 instead of no CFG (ablation) | class-conditioned GaussianCube generation · OmniObject3D | FID-50K: 13.39 -> 11.62 | Helped |
| increase classifier-free guidance scale from 2.0 to 6.0 (ablation) | class-conditioned GaussianCube generation · OmniObject3D | FID-50K: 11.62 -> 32.80 | Hurt |
Surprising findings
- Naive voxel grid with learnable offsets and no densification fails badly at fitting, indicating densification matters for fitting quality.
- The paper reports that Shap-E and LGM achieve comparable CLIP scores, but they use millions of training data or a 2D diffusion prior.
Limitations
- Constructing each GaussianCube takes around 5 minutes per object, which hinders scaling to large 3D datasets.
- The text-to-3D model struggles to generate the exact number of objects specified in prompts.
- Uses an approximate OT solver (segment-wise Jonker-Volgenant) because exact JV is O(N^3).
Experimental setup
Datasets: ShapeNet Car ShapeNet Chair OmniObject3D Synthetic Avatar dataset Objaverse
Benchmarks: ShapeNet Car ShapeNet Chair OmniObject3D Synthetic Avatar Objaverse
Metrics: PSNR LPIPS SSIM FID-50K KID-50K CSIM FID-5K KID-5K CLIP score inference time
Baselines: EG3D GET3D DiffTF Rodin DreamGaussian VolumeDiffusion Shap-E LGM Triplane Voxels Instant-NGP
Builds on: 3D Gaussian Splatting ADM U-Net DPM-solver Jonker-Volgenant algorithm
Closest papers by meaning
- High-fidelity 3D Object Generation from Single Image with RGBN-Volume Gaussian Reconstruction Model (CVPR 2025 · similarity 0.89)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.88)
- XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies (CVPR 2024 · similarity 0.88)
- HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel View Synthesis (NeurIPS 2025 · similarity 0.88)
- DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (CVPR 2024 · similarity 0.88)
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.