Research map / 3D vision / 3D generation
DiffInDScene: Diffusion-based High-Quality 3D Indoor Scene Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DiffInDScene, a cascaded sparse 3D diffusion pipeline with a multi-scale occupancy autoencoder and stochastic TSDF fusion that generates room-level indoor geometry from scratch and refines MVS reconstructions.
Problem
Room-level 3D indoor scene geometry generation is computationally infeasible for standard diffusion models because TSDF volumes are large and sparse, and existing scene methods (e.g., Text2Room) produce fragmented, distorted meshes.
Approach
A three-stage coarse-to-fine cascade of sparse diffusion models that denoise only occupied voxels, conditioned on latent occupancy codes from a multi-scale Patch-VQGAN, plus a stochastic TSDF fusion that diffuses overlapping local crops and fuses them at each timestep to produce a whole-room TSDF.
Key results
- On generated scenes, mean circularity 0.781 and shape regularity 0.816 versus Text2Room's 0.674 and 0.716; user completeness score 4.856 vs 2.532.
- Refining NeuralRecon with DiffInDScene lowers the <45° mean normal error from 10.27 to 8.09 and raises the <45° inlier ratio from 59.97% to 65.05%.
- In the 10-scene user study on ScanNet refinement, NR+Ours scored 87.14 overall, above ground-truth meshes (85.62).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| generate scenes with DiffInDScene instead of Text2Room | 3D-FRONT scene generation · 3D-FRONT | Completeness: 2.532 -> 4.856 | Helped |
| add Poisson reconstruction to Text2Room | Text2Room · 3D-FRONT | Completeness: 2.532 -> 3.228 | Helped |
| refine NeuralRecon output with DiffInDScene | NeuralRecon · ScanNet | < 30° normal ratio: 52.88% -> 59.27% | Helped |
| use sparse rather than dense diffusion (ablation) | DiffInDScene diffusion model · ScanNet TSDF crops | GPU memory: 22.8 -> 11.8 | Helped |
| use average fusion rather than stochastic fusion (ablation) | DiffInDScene final-stage diffusion · ScanNet scene generation | mesh quality | Hurt |
| use stochastic fusion rather than independent crop diffusion (ablation) | DiffInDScene final-stage diffusion · ScanNet scene generation | global consistency | Helped |
Surprising findings
- User study ranked refined meshes from DiffInDScene above ScanNet ground-truth meshes on overall quality.
- Sparse diffusion used 0.008 TFLOPs versus 3.290 TFLOPs for dense at batch size 1 with the same parameter count.
Limitations
- Generated geometry drifts from ground truth because the model operates on a relaxed occupancy, so correspondence-based metrics are unsuitable.
- Evaluation is small: 11 scenes for objective metrics and 5 scenes per method in the generation user study.
- Texture for visualizations is produced by a separate method (DreamSpace), not by DiffInDScene itself.
Experimental setup
Datasets: 3D-FRONT ScanNet
Benchmarks: ScanNet test split
Metrics: aspect ratio circularity shape regularity user study completeness and perceptual quality normal error user ranking
Baselines: Text2Room Text2Room + Poisson NeuralRecon NeuralRecon + Laplacian denoising SimpleRecon
Builds on: DDPM VQ-VAE-2 KinectFusion TorchSparse
Closest papers by meaning
- DiffuScene: Denoising Diffusion Models for Generative Indoor Scene Synthesis (CVPR 2024 · similarity 0.90)
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.88)
- Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting (NeurIPS 2024 · similarity 0.87)
- DeBaRA: Denoising-Based 3D Room Arrangement Generation (NeurIPS 2024 · similarity 0.87)
- RoomPainter: View-Integrated Diffusion for Consistent Indoor Scene Texturing (CVPR 2025 · similarity 0.87)
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.