Research map / 3D vision / 3D generation
XCube: Large-Scale 3D Generative Modeling using Sparse Voxel Hierarchies
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes XCube, a hierarchical sparse-voxel latent diffusion model (built on VDB) that generates high-resolution (up to 1024^3) 3D objects and large driving scenes with arbitrary attributes in a feed-forward, coarse-to-fine manner.
Problem
Generating high-resolution 3D shapes and large-scale outdoor scenes with fine detail is hard for existing 3D generative priors, which are limited in resolution, scalability, and efficiency, and often require slow test-time optimization.
Approach
A cascade of latent diffusion models over a sparse voxel hierarchy, where each level's sparse structure VAE latent is generated conditioned on the coarser level, combined with a custom sparse 3D learning framework on VDB for efficient high-resolution processing.
Key results
- On ShapeNet 1-NNA, XCube achieves CD 52.85 / EMD 49.75 on Airplane and 53.99 / 48.60 on Chair, best among compared methods (e.g., LION 65.10 / 60.15 on Airplane).
- In a user study on Objaverse text-to-3D against Shap-E, 79.2% of 900 pairwise comparisons favored XCube.
- On Waymo scenes, 66.3% of 900 pairwise comparisons favored generated scenes over validation ground truth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Airplane | 1-NNA CD (%): 57.55 -> 52.85 | Helped |
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Airplane | 1-NNA EMD (%): 53.47 -> 49.75 | Helped |
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Chair | 1-NNA CD (%): 54.06 -> 53.99 | Helped |
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Chair | 1-NNA EMD (%): 54.06 -> 48.60 | Helped |
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Car | 1-NNA CD (%): 60.61 -> 57.96 | Helped |
| use XCube instead of the strongest listed baseline | 3D generative models · ShapeNet Car | 1-NNA EMD (%): 54.94 -> 54.43 | Helped |
| replace progressive pruning with a single pruning step (ablation) | XCube VAE on ShapeNet Chairs · ShapeNet Chairs | grid IoU: 92.88% -> 89.68% | Hurt |
| add progressive pruning (ablation) | XCube VAE on ShapeNet Chairs · ShapeNet Chairs | GPU memory usage: 3× | Helped |
| replace the single-level hierarchy with 163 → 1283 → 5123 (ablation) | XCube · ShapeNet Chairs | 1-NNA CD (%): 59.31 -> 53.99 | Helped |
| replace the single-level hierarchy with 163 → 1283 → 5123 (ablation) | XCube · ShapeNet Chairs | 1-NNA EMD (%): 57.46 -> 48.60 | Helped |
| use 323 rather than 163 as the initial hierarchy resolution (ablation) | XCube · ShapeNet Chairs | 1-NNA CD (%): 53.99 -> 55.39 | Hurt |
| use 323 rather than 163 as the initial hierarchy resolution (ablation) | XCube · ShapeNet Chairs | 1-NNA EMD (%): 48.60 -> 51.40 | Hurt |
Surprising findings
- Generated Waymo scenes were preferred over real validation ground truth in 66.3% of user comparisons.
Limitations
- Text-to-3D model struggles with complex prompts due to limited 3D datasets compared to image datasets.
- Error accumulation across hierarchy levels in the factorized modeling, mitigated but not eliminated by a refinement network.
Experimental setup
Datasets: ShapeNet Objaverse Waymo Open Dataset Karton City Cap3D captions
Benchmarks: ShapeNet Airplane/Chair/Car 1-NNA
Metrics: 1-NNA CD 1-NNA EMD reconstruction grid IoU user preference rate
Baselines: PVD LION NFD NWD LAS-Diffusion 3DShape2VecSet Shap-E
Builds on: latent diffusion hierarchical latent diffusion v-parameterization DDIM VDB
Closest papers by meaning
- GaussianCube: A Structured and Explicit Radiance Representation for 3D Generative Modeling (NeurIPS 2024 · similarity 0.88)
- HyperCube: Implicit Field Representations of Voxelized 3D Models (Student Abstract) (AAAI 2024 · similarity 0.87)
- LT3SD: Latent Trees for 3D Scene Diffusion (CVPR 2025 · similarity 0.86)
- Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering (CVPR 2025 · similarity 0.86)
- MeshFormer : High-Quality Mesh Generation with 3D-Guided Reconstruction Model (NeurIPS 2024 · similarity 0.85)
- SCube: Instant Large-Scale Scene Reconstruction using VoxSplats (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.