Research map / 3D vision / Gaussian splatting
CompGS: Unleashing 2D Compositionality for Compositional Text-to-3D via Dynamically Optimizing 3D Gaussians
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CompGS, a 3D Gaussian Splatting framework for compositional text-to-3D that initializes entity Gaussians from a 2D compositional image and optimizes them with dynamic SDS (decomposed entity/composition levels plus volume-adaptive scaling).
Problem
Compositional text-to-3D generation, where multiple objects with attributes, relations and interactions must be generated coherently in one 3D scene; SDS-based optimization struggles with multi-object consistency and scale variation.
Approach
Transfer 2D compositionality by generating a compositional image, segmenting it per entity (LLM + LangSAM), lifting each entity with an image-to-3D model (TripoSR) to initialize Gaussians, then alternating entity-level and composition-level SDS with gradient masking by bounding box and volume normalization of each entity during optimization.
Key results
- On T3Bench multiple-objects track, COMPGS achieves average 46.1 (quality 54.2, alignment 37.9), outperforming VP3D (40.3) and ProlificDreamer (35.8).
- User study ranking score 7.23 for COMPGS vs 6.71 for VP3D (higher is better in the reported table).
- Ablation on T3Bench: removing Gaussian initialization drops average from 45.9 to 20.8; removing decomposed optimization drops it to 41.0.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Gaussian initialization from 2D compositionality instead of random initialization (ablation) | COMPGS · T3Bench | Quality score: 22.8 -> 53.8 | Helped |
| Use Gaussian initialization from 2D compositionality instead of random initialization (ablation) | COMPGS · T3Bench | Alignment score: 18.7 -> 38.0 | Helped |
| Use decomposed entity-level and composition-level optimization (ablation) | COMPGS · T3Bench | Quality score: 46.8 -> 53.8 | Helped |
| Use decomposed entity-level and composition-level optimization (ablation) | COMPGS · T3Bench | Alignment score: 35.2 -> 38.0 | Helped |
| Use volume-adaptive optimization (ablation) | COMPGS · T3Bench | Quality score: 50.8 -> 53.8 | Helped |
| Use volume-adaptive optimization (ablation) | COMPGS · T3Bench | Alignment score: 36.4 -> 38.0 | Helped |
| Use COMPGS rather than VP3D | Compositional text-to-3D generation · T3Bench multiple objects | Quality score: 49.1 -> 54.2 (+5.1) | Helped |
| Use COMPGS rather than VP3D | Compositional text-to-3D generation · T3Bench multiple objects | Alignment score: 31.5 -> 37.9(+6.4) | Helped |
| Use COMPGS rather than VP3D | Compositional text-to-3D generation · User study | Ranking score: 6.71 -> 7.23 | Helped |
Surprising findings
- Training with 5k iterations already produces plausible 3D assets, close to the 10k-iteration result.
Limitations
- Compositional generation takes longer than single-object generation, and runtime grows with the number of objects.
- Relies on a pipeline of external models (T2I, LLM, segmentation, image-to-3D), so errors in these components propagate.
Experimental setup
Datasets: T3Bench
Benchmarks: T3Bench
Metrics: CLIP quality score ROUGE-L caption recall alignment user study ranking
Baselines: DreamFusion Magic3D Latent-NeRF Fantasia3D SJC ProlificDreamer VP3D Set-the-Scene OpenLRM TripoSR GALA3D GraphDreamer
Builds on: DreamFusion 3D Gaussian Splatting MVDream ThreeStudio TripoSR
Closest papers by meaning
- GALA3D: Towards Text-to-3D Complex Scene Generation via Layout-guided Generative Gaussian Splatting (ICML 2024 · similarity 0.91)
- Text-to-3D using Gaussian Splatting (CVPR 2024 · similarity 0.89)
- GaussianDreamer: Fast Generation from Text to 3D Gaussians by Bridging 2D and 3D Diffusion Models (CVPR 2024 · similarity 0.88)
- DreamGaussian: Generative Gaussian Splatting for Efficient 3D Content Creation (ICLR 2024 · similarity 0.88)
- DiffSplat: Repurposing Image Diffusion Models for Scalable Gaussian Splat Generation (ICLR 2025 · similarity 0.88)
- Align Your Gaussians: Text-to-4D with Dynamic 3D Gaussians and Composed Diffusion Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.