Research map / 3D vision / 3D generation
MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MMGDreamer, a dual-branch diffusion model that generates 3D indoor scenes from a Mixed-Modality Graph whose nodes carry text, image, or both, with optional edges, plus a visual enhancement module and relation predictor.
Problem
Text-only scene graphs cannot precisely specify object geometry and assume complete relation annotations, limiting controllable 3D indoor scene generation from flexible user inputs.
Approach
A Mixed-Modality Graph whose nodes can hold text, images, or both and whose edges are optional; a VQ-VAE-style visual enhancement module synthesizes visual features for text-only nodes, and a GCN-based relation predictor infers missing edges before a dual-branch (layout and shape) diffusion model generates the scene.
Key results
- Living room FID reduced from 75.95 (EchoScene) to 68.94 with MM+R, about 9% relative improvement.
- Ablation: VEM plus RP gives FID 41.84 and KID 2.55 with mSG 0.86, versus baseline FID 45.10 and mSG 0.63.
- Object-level MMD and COV improved over EchoScene across all listed object categories.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MMGDreamer (MM+R) instead of EchoScene | 3D indoor scene generation · Living room | FID: 75.95 -> 68.94 | Helped |
| Use MMGDreamer (MM+R) instead of EchoScene | 3D indoor scene generation · Living room | FID_CLIP: 6.73 -> 6.19 | Helped |
| Use MMGDreamer (MM+R) instead of EchoScene | 3D indoor scene generation · Living room | KID: 0.60 -> 0.40 | Helped |
| Add the visual enhancement module (VEM) (ablation) | MMGDreamer · SG-FRONT scene generation | FID: 45.10 -> 43.50 | Helped |
| Add the visual enhancement module (VEM) (ablation) | MMGDreamer · SG-FRONT scene generation | KID: 3.74 -> 3.27 | Helped |
| Add the relation predictor (RP) (ablation) | MMGDreamer · SG-FRONT scene generation | mSG: 0.63 -> 0.83 | Helped |
| Add both VEM and RP (ablation) | MMGDreamer · SG-FRONT scene generation | FID: 45.10 -> 41.84 | Helped |
| Add both VEM and RP (ablation) | MMGDreamer · SG-FRONT scene generation | KID: 3.74 -> 2.55 | Helped |
| Add both VEM and RP (ablation) | MMGDreamer · SG-FRONT scene generation | mSG: 0.63 -> 0.86 | Helped |
Limitations
- Generated scenes occasionally show object interpenetration inherited from noisy 3D-FRONT data.
- Textures and materials are excluded; only geometry is modeled.
- Object-level experiments use a relatively small amount of training data.
Experimental setup
Datasets: SG-FRONT 3D-FRONT 3D-FUTURE
Benchmarks: SG-FRONT
Metrics: FID FID_CLIP KID MMD COV 1-NNA mSG Chamfer Distance
Baselines: Graph-to-3D CommonScenes EchoScene
Builds on: EchoScene CommonScenes Triplet-GCN latent diffusion model CLIP
Closest papers by meaning
- FreeScene: Mixed Graph Diffusion for 3D Scene Synthesis from Free Prompts (CVPR 2025 · similarity 0.89)
- InstructScene: Instruction-Driven 3D Indoor Scene Synthesis with Semantic Graph Prior (ICLR 2024 · similarity 0.88)
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.88)
- ControlRoom3D: Room Generation using Semantic Proxy Rooms (CVPR 2024 · similarity 0.88)
- GraphDreamer: Compositional 3D Scene Synthesis from Scene Graphs (CVPR 2024 · similarity 0.88)
- SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.