Research map / 3D vision / 3D generation
SceneX: Procedural Controllable Large-Scale Scene Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SceneX, an LLM-agent framework that turns text descriptions into large-scale 3D scenes and assets in Blender by combining a curated procedural-asset hub (PCGHub) with a multi-stage planner (PCGPlanner).
Problem
Procedural controllable generation (PCG) for large-scale 3D scenes requires expert knowledge, offers limited modular diversity, and struggles to coordinate the many elements of complex scenes.
Approach
Encapsulate Blender procedural plugins as documented APIs (PCGHub) and let a GPT-4-driven multi-stage planner (scene decomposition, terrain, object generation/retrieval, asset placement) map user text to executable Blender actions, with a structured prompt template (role, task, document, format, examples).
Key results
- Aesthetic evaluation by 35 volunteers and 5 experts gives SceneX AS 7.83 and AES 7.70, above the next-best method by 2.10 and 0.9 points.
- Generating a 150m x 150m scene takes 2 min versus 14 min for Infinigen, and a city of 2.5km x 2.5km takes about 20 h versus over 3 weeks for a human expert.
- GPT-4 achieves the best ER@1/SR@1 of 86.00/86.05 on scene generation and 96.00/85.42 on asset generation among the tested LLMs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Task and Document together rather than Document alone (ablation) | SceneX tree asset generation prompt · Tree asset generation | ER@1: 42.00 -> 84.00 | Helped |
| Use Task and Document together rather than Document alone (ablation) | SceneX tree asset generation prompt · Tree asset generation | SR@1: 47.62 -> 71.43 | Helped |
| Add Examples to the Document prompt (ablation) | SceneX tree asset generation prompt · Tree asset generation | ER@1: 42.00 -> 92.00 | Helped |
| Add Examples to the Document prompt (ablation) | SceneX tree asset generation prompt · Tree asset generation | SR@1: 47.62 -> 76.09 | Helped |
| Add Task to the Examples and Document prompt (ablation) | SceneX tree asset generation prompt · Tree asset generation | SR@1: 76.09 -> 78.26 | Helped |
| Add Role to the Task, Document, and Examples prompt (ablation) | SceneX tree asset generation prompt · Tree asset generation | SR@1: 78.26 -> 80.85 | Helped |
| Remove Role from the full prompt (ablation) | SceneX tree asset generation prompt · Tree asset generation | ER@1: 94.00 -> 92.00 | No clear effect |
Surprising findings
- Mistral (open-source) is reported as the second-best LLM, close behind GPT-4, and outperforms GPT-3.5-turbo on executability for scene generation (76.00 vs 66.00 ER@1).
- On CLIP similarity, text-to-3D methods such as DreamFusion score higher than SceneX, which the authors attribute to their text-image alignment optimization.
Limitations
- Dependence on a pretrained LLM may restrict generalization across applications.
- Constrained by the inherent algorithms and rules of existing PCG modules.
- Evaluation relies on a small self-constructed prompt set and subjective volunteer scoring.
Experimental setup
Datasets: PCGHub (172 procedural assets, 263 APIs, 11,284 static 3D assets) GPT-4-generated scene and asset descriptions (50 scene, 50 asset, 20 editing)
Benchmarks: Self-built 50 scene / 50 asset description set
Metrics: ER@1 SR@1 average score (AS) average expert score (AES) CLIP similarity generation time
Baselines: Magic 3D DreamFusion Text2Room CityDreamer Infinigen 3D-GPT WonderJ SceneCraft BlenderGPT SceneDreamer InfiniCity PersistentNature
Builds on: Infinigen 3D-GPT SceneCraft CLIP
Closest papers by meaning
- X-Scene: Large-Scale Driving Scene Generation with High Fidelity and Flexible Controllability (NeurIPS 2025 · similarity 0.87)
- SceneCraft: An LLM Agent for Synthesizing 3D Scenes as Blender Code (ICML 2024 · similarity 0.86)
- WonderWorld: Interactive 3D Scene Generation from a Single Image (CVPR 2025 · similarity 0.85)
- WorldAct: Activating Monolithic 3D Worlds into Interactive-Ready Object-Centric Scenes (NeurIPS 2026 · similarity 0.85)
- WorldCoder-Bench: Benchmarking Physically Grounded 3D World Synthesis (NeurIPS 2026 · similarity 0.84)
- Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.