Research map / Large language models / Alignment and preferences
OptiScene: LLM-driven Indoor Scene Layout Generation via Scaled Human-aligned Data Synthesis and Multi-Stage Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds 3D-SynthPlace (about 16.7k indoor layouts combining 3D-Front and filtered Holodeck outputs) and OptiScene, a LoRA-tuned Qwen3-8B that uses SFT with semantic-summary reasoning followed by two-stage DPO for LLM-based indoor layout generation.
Problem
Indoor layout generation suffers from spatial inconsistency and cost in prompt-driven GPT pipelines, and from small, bedroom/living-room-centric datasets and lack of human-preference alignment in learning-based methods.
Approach
Fine-tune an open-source LLM on a larger, expanded layout dataset with a natural-language semantic-summary reasoning step before predicting object placements, then align it with human preferences using two DPO stages: human-consensus positives versus model samples, and synthetic geometric-violation hard negatives.
Key results
- OptiScene reaches FID 26.45 and OOR 0.01 on the 3D-SynthPlace bedroom set, versus 37.66 / 0.04 for LLplace (w/ 3D-Synth).
- Ablation shows usability rising from 28% (SFT only) to 50% after both DPO stages, with OOR falling from 0.049 to 0.023.
- Room editing reaches 90% editing success rate on 20 cases, per the paper's text.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OptiScene instead of LLplace (w/ 3D-Synth) | Indoor layout generation · 3D-SynthPlace bedroom | FID: 37.66 -> 26.45 | Helped |
| Use OptiScene instead of LLplace (w/ 3D-Synth) | Indoor layout generation · 3D-SynthPlace living room | OOR: 0.08 -> 0.02 | Helped |
| Use OptiScene instead of LLplace (w/ 3D-Synth) | Indoor layout generation · 3D-SynthPlace kitchen | FID: 43.68 -> 41.73 | Helped |
| Use OptiScene instead of LLplace (w/ 3D-Synth) | Indoor layout generation · 3D-SynthPlace bathroom | OOR: 0.03 -> 0.02 | Helped |
| Add high-level semantic reasoning to SFT (ablation) | OptiScene SFT · Ablation study | Usability Rate: 28% -> 33% | Helped |
| Add high-level semantic reasoning to SFT (ablation) | OptiScene SFT · Ablation study | OOR: 0.049 -> 0.048 | Helped |
| Add DPO-stage I to SFT with reasoning (ablation) | OptiScene · Ablation study | OOR: 0.048 -> 0.033 | Helped |
| Add DPO-stage I to SFT with reasoning (ablation) | OptiScene · Ablation study | Usability Rate: 33% -> 40% | Helped |
| Add DPO-stage II to SFT with reasoning and DPO-stage I (ablation) | OptiScene · Ablation study | OOR: 0.033 -> 0.023 | Helped |
| Add DPO-stage II to SFT with reasoning and DPO-stage I (ablation) | OptiScene · Ablation study | Usability Rate: 40% -> 50% | Helped |
| Use high-level semantic descriptions instead of scene-graph reasoning supervision (ablation) | OptiScene reasoning supervision · Room layout generation | Layout quality | Helped |
Surprising findings
- Adding the semantic reasoning step to SFT barely changed OOR (0.049 to 0.048), with the usability gain coming mainly from DPO.
Limitations
- Only single-room layouts are generated, not whole houses or buildings.
- The dataset covers furniture only, without small objects or wall objects.
- Dataset counts and sample sizes are inconsistent across the paper (e.g., 12,000 vs 9,360 synthetic layouts; 1,100+1,100 vs 2,200 DPO positives).
Experimental setup
Datasets: 3D-Front Holodeck-Synth Objaverse 3D-SynthPlace
Benchmarks: 3D-Front test split 3D-SynthPlace extended test set
Metrics: FID object overlap rate GPT-4o ratings (functionality, layout, aesthetics) usability rate generation success rate editing success rate navigation success rate
Baselines: I-Design DiffuScene LLplace Holodeck DiffuScene (w/ 3D-Synth) LLplace (w/ 3D-Synth)
Builds on: Direct Preference Optimization Holodeck LayoutGPT DiffuScene LLplace
Closest papers by meaning
- Direct Numerical Layout Generation for 3D Indoor Scene Synthesis via Spatial Reasoning (NeurIPS 2025 · similarity 0.92)
- DeBaRA: Denoising-Based 3D Room Arrangement Generation (NeurIPS 2024 · similarity 0.90)
- Hierarchically-Structured Open-Vocabulary Indoor Scene Synthesis with Pre-trained Large Language Model (AAAI 2025 · similarity 0.89)
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.88)
- HouseLayout3D: A Benchmark and Training-free Baseline for 3D Layout Estimation in the Wild (NeurIPS 2025 · similarity 0.88)
- CASAGPT: Cuboid Arrangement and Scene Assembly for Interior Design (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.