Research map / 3D vision / 3D generation
ControlRoom3D: Room Generation using Semantic Proxy Rooms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates textured 3D room meshes from a user-defined 3D semantic proxy room (bounding boxes) plus a text style prompt, using guided panorama generation, depth alignment to boxes, and mesh completion.
Problem
Text-only 3D room mesh generation often yields implausible layouts (e.g., duplicated beds) and lacks user control over room composition.
Approach
Render a user-defined 3D semantic bounding-box proxy room to 2D semantic, instance and depth maps and use them to control a latent diffusion model (semantic and depth adapters, correspondence-aware attention) for panorama generation, then align monocular metric depth to the boxes and iteratively complete the mesh.
Key results
- ControlRoom3D achieves CLIP score 28.4 vs 24.0 (Text2Room) and 25.6 (MVDiffusion).
- User study layout plausibility 4.54 for ControlRoom3D vs 2.12 (Text2Room) and 3.01 (MVDiffusion).
- Ablation: geometry alignment gives +2.32 proxy alignment; full pipeline reaches 4.22 3DS and 4.23 PQ.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add a 3D semantic proxy room (ablation) | ControlRoom3D · Room generation | 3D Structure Completeness: 1.78 -> 2.81 | Helped |
| add geometry alignment (ablation) | ControlRoom3D with a semantic proxy room · Room generation | Proxy Alignment: 2.02 -> 4.34 | Helped |
| add panorama generation (ablation) | ControlRoom3D with a proxy room and geometry alignment · Room generation | Perceptual Quality: 3.45 -> 3.95 | Helped |
| add mesh completion (ablation) | ControlRoom3D with a proxy room, geometry alignment, and panorama generation · Room generation | 3D Structure Completeness: 3.68 -> 4.22 | Helped |
| use ControlRoom3D instead of Text2Room | Text2Room · Room generation | Layout Plausibility: 2.12 -> 4.54 | Helped |
| use ControlRoom3D instead of MVDiffusion | MVDiffusion · Room generation | Perceptual Quality: 2.89 -> 4.07 | Helped |
| use ControlRoom3D instead of MVDiffusion | MVDiffusion · Room generation | Inception Score: 6.37 -> 5.75 | Hurt |
Surprising findings
- Inception score is highest for Text2Room (6.47) and lowest for ControlRoom3D (5.75), which the authors attribute to IS favoring diversity over plausible layouts.
Limitations
- Text-only baselines cannot produce layouts consistent with proxy rooms, so comparisons are adapted rather than direct.
- Inception score favors diversity and does not reflect layout plausibility.
Experimental setup
Datasets: HyperSim Matterport3D
Benchmarks: user study (3D structure completeness, layout plausibility, perceptual quality) CLIP score Inception score
Metrics: CLIP score Inception score 3D structure completeness layout plausibility perceptual quality proxy alignment
Baselines: Text2Room MVDiffusion
Builds on: MVDiffusion Text2Room ZoeDepth Segment Anything ControlNet
Closest papers by meaning
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.89)
- MMGDreamer: Mixed-Modality Graph for Geometry-Controllable 3D Indoor Scene Generation (AAAI 2025 · similarity 0.88)
- DeBaRA: Denoising-Based 3D Room Arrangement Generation (NeurIPS 2024 · similarity 0.87)
- OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization (NeurIPS 2025 · similarity 0.86)
- Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning (NeurIPS 2025 · similarity 0.86)
- SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.