Research map / 3D vision / 3D generation
GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
GALA3D uses LLM-generated coarse layouts to drive a layout-guided 3D Gaussian Splatting representation, optimized with instance-level MVDream SDS, ControlNet-conditioned scene-level diffusion, and a learnable layout refinement for compositional text-to-3D scene generation.
Problem
Text-to-3D generation of complex multi-object scenes with correct object interactions, spatial layout, and multi-view consistency, where shared 3D representations and LLM-produced layouts often fail.
Approach
Introduce layout-guided 3D Gaussians (per-instance Gaussians placed by LLM-derived layout boxes), adaptive geometry control (distribution and shape regularization), and jointly refine the LLM layout during diffusion-based optimization with instance-level multi-view SDS and scene-level ControlNet SDS.
Key results
- Average CLIP Score of 34.573 across 22 scenes, versus 31.174 for the next best method LucidDreamer and 30.856 for MVDream.
- User study (125 participants) gives GALA3D 8.42 scene quality and 9.68 scene consistency, above all baselines including Set-the-scene.
- Ablation: removing adaptive geometry control drops CLIP score from 34.885 (full) to 32.198.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GALA3D instead of LucidDreamer | Text-to-3D generation · Average across 22 scenes | CLIP Score: 31.174 -> 34.573 | Helped |
| Use GALA3D instead of ProlificDreamer | Text-to-3D generation · Case 1 | CLIP Score: 30.237 -> 31.637 | Helped |
| Use GALA3D instead of MVDream | Text-to-3D generation · Case 2 | CLIP Score: 32.636 -> 37.658 | Helped |
| Use GALA3D instead of GSGEN | Text-to-3D generation · Case 3 | CLIP Score: 29.959 -> 31.459 | Helped |
| Use GALA3D instead of MVDream | Text-to-3D generation · Case 4 | CLIP Score: 27.417 -> 35.052 | Helped |
| Add Adaptive Geometry Control (ablation) | GALA3D · Text-to-3D scenes | CLIP Score: 32.198 -> 34.885 | Helped |
| Add Layout Refinement Module (ablation) | GALA3D · Text-to-3D scenes | CLIP Score: 34.293 -> 34.885 | Helped |
| Add Compositional Optimization Scheme (ablation) | GALA3D · Text-to-3D scenes | CLIP Score: 32.213 -> 34.885 | Helped |
| Add global loss (ablation) | GALA3D · Text-to-3D scenes | CLIP Score: 34.342 -> 34.885 | Helped |
| Add layout loss (ablation) | GALA3D · Text-to-3D scenes | CLIP Score: 33.297 -> 34.885 | Helped |
Limitations
- Relies on CLIP Score as the main metric since no ground truth exists for zero-shot generation.
- Many compositional NeRF comparisons could not be run because those methods are not open-source; results were taken from papers.
Experimental setup
Benchmarks: GALA3D 22-scene text prompt set (1 to 10 objects)
Metrics: CLIP Score user study scores (scene quality, geometric fidelity, text a
Baselines: Latent-NeRF ProlificDreamer MVDream SJC DreamGaussian GaussianDreamer GSGEN LucidDreamer Set-the-scene
Builds on: 3D Gaussian Splatting DreamFusion MVDream ControlNet Set-the-scene
Closest papers by meaning
- CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussians (CVPR 2025 · similarity 0.91)
- Layout-your-3D: Controllable and Precise 3D Generation with 2D Blueprint (ICLR 2025 · similarity 0.90)
- Text-to-3D using Gaussian Splatting (CVPR 2024 · similarity 0.89)
- Towards Text-guided 3D Scene Composition (CVPR 2024 · similarity 0.88)
- 3DitScene: Editing Any Scene via Language-guided Disentangled Gaussian Splatting (ICLR 2025 · similarity 0.88)
- Hierarchical Gaussian Mixture Model Splatting for Efficient and Part Controllable 3D Generation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.