Research map / 3D vision / 3D generation
LT3SD: Latent Trees for 3D Scene Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LT3SD, a latent-tree-based latent diffusion model that generates large, arbitrary-sized 3D indoor scenes patch-by-patch in a coarse-to-fine hierarchy.
Problem
Generating large-scale, high-fidelity, arbitrary-sized 3D scenes, which existing 3D diffusion models (designed for bounded object-level shapes) do not handle well.
Approach
Decompose a TUDF voxel scene into a hierarchical latent tree where each level has a coarse geometry grid and a latent feature grid; train per-level diffusion models on random scene patches conditioned on the geometry, and generate infinite scenes via inpainting-based patch generation and MultiDiffusion-style parallel denoising.
Key results
- Achieves FID 13.39 versus 45.55 for BlockFusion, 237.85 for PVD and 266.27 for NFD on 3D-FRONT unconditional generation (about 70% FID improvement).
- Three-level latent tree reaches COV of 53.10% (CD) versus 43.82% for PVD and 24.32% for BlockFusion.
- Latent tree lowers reconstruction error to 3.20e-4 from 4.91e-4 for latent cascaded models at the same parameter count (13.3M).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the three-level latent tree (17.6 - 8.8 - 2.2 cm) instead of either reported single-level configuration (ablation) | LT3SD · 3D-FRONT test | FID: 59.23 -> 13.39; 50.07 -> 13.39 | Helped |
| use the three-level latent tree (17.6 - 8.8 - 2.2 cm) instead of either reported single-level configuration (ablation) | LT3SD · 3D-FRONT test | COV (CD, %): 28.61 -> 53.10; 40.87 -> 53.10 | Helped |
| factorize scenes into a latent tree rather than use latent CM (ablation) | LT3SD scene autoencoder · 3D-FRONT test reconstruction | Reconstruction error (ℓ2): 4.91e-4 -> 3.20e-4 | Helped |
| use the latent tree representation instead of latent CM (ablation) | LT3SD scene autoencoder · 3D-FRONT test | training time: × 1.00 -> × 0.87 | Helped |
| use the latent tree representation instead of latent CM (ablation) | LT3SD scene autoencoder · 3D-FRONT latent storage | storage: × 1.00 -> × 0.80 | Helped |
| use LT3SD instead of BlockFusion | 3D scene generation · 3D-FRONT test | FID: 45.55 -> 13.39 | Helped |
| use LT3SD instead of PVD | 3D scene generation · 3D-FRONT test | COV (CD, %): 43.82 -> 53.10 | Helped |
| use LT3SD instead of NFD | 3D scene generation · 3D-FRONT test | MMD (CD, ×1e2): 3.65 -> 3.51 | Helped |
Surprising findings
- BlockFusion achieves a better FID than PVD and NFD but worse point-cloud coverage and distance metrics, suggesting FID and point-cloud metrics capture different aspects of scene quality.
Limitations
- Trained only on indoor house-level data from 3D-FRONT without semantic annotations.
- Unconditional generation is the main evaluation; completion is qualitative only.
Experimental setup
Datasets: 3D-FRONT
Benchmarks: 3D-FRONT
Metrics: FID COV MMD 1-NNA Chamfer Distance EMD reconstruction error (l2)
Baselines: PVD NFD BlockFusion Latent cascaded models
Builds on: latent diffusion models MultiDiffusion denoising diffusion probabilistic models
Closest papers by meaning
- A 3D Scene is Worth 1K Tokens: 3D-Grounded Representation for Scene Generation at Scale (NeurIPS 2026 · similarity 0.94)
- Prometheus: 3D-Aware Latent Diffusion Models for Feed-Forward Text-to-3D Scene Generation (CVPR 2025 · similarity 0.92)
- Denoising Diffusion via Image-Based Rendering (ICLR 2024 · similarity 0.90)
- PartCrafter: Structured 3D Mesh Generation via Compositional Latent Diffusion Transformers (NeurIPS 2025 · similarity 0.90)
- SceneFactor: Factored Latent 3D Diffusion for Controllable 3D Scene Generation (CVPR 2025 · similarity 0.90)
- Diffusion$^2$: Dynamic 3D Content Generation via Score Composition of Video and Multi-view Diffusion Models (ICLR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.