Research map / 3D vision / 3D generation
PhyScene: Physically Interactable 3D Scene Synthesis for Embodied AI
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PhyScene, a conditional diffusion model for scene layout generation with physics- and interactivity-based guidance (collision, room layout, reachability) to produce physically interactable 3D indoor scenes with articulated objects.
Problem
Existing 3D indoor scene synthesis methods focus on realism and neglect physical plausibility (object collisions, layout violations) and interactivity (object reachability for embodied agents), which limits their use for embodied AI simulation.
Approach
Train a conditional diffusion model on floor-plan-conditioned scene layouts and, at inference, tilt the denoising transition with gradients of differentiable guidance functions: 3D-IoU-based collision and room-boundary penalties plus an A*-path-based agent reachability penalty; articulated objects are retrieved via shape-feature matching to GAPartNet.
Key results
- On unconditioned bedroom synthesis, PhyScene reduces object collision rate Colobj to 0.187 vs 0.238 for DiffuScene and 0.255 for ATISS.
- In floor-conditioned living room synthesis, PhyScene lowers Colscene to 0.63 vs 0.85 (ATISS) and 0.69 (DiffuScene).
- Ablation shows collision guidance lowers Colobj to 0.111 alone, while combined guidance balances collision and layout metrics (Colobj 0.191, Rout 0.219).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add collision guidance (ablation) | PhyScene scene generation · Living Room | Colobj: 0.200 -> 0.111 | Helped |
| add collision guidance (ablation) | PhyScene scene generation · Living Room | Rout: 0.240 -> 0.354 | Hurt |
| add room-layout guidance (ablation) | PhyScene scene generation · Living Room | Rout: 0.240 -> 0.110 | Helped |
| add room-layout guidance (ablation) | PhyScene scene generation · Living Room | Colobj: 0.200 -> 0.279 | Hurt |
| add agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Rwalkable: 0.808 -> 0.927 | Helped |
| add agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Rreach: 0.763 -> 0.813 | Helped |
| combine collision, room-layout, and agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Colobj: 0.200 -> 0.191 | Helped |
| combine collision, room-layout, and agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Rout: 0.240 -> 0.219 | Helped |
| combine collision, room-layout, and agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Rwalkable: 0.808 -> 0.815 | Helped |
| combine collision, room-layout, and agent-interactiveness guidance (ablation) | PhyScene scene generation · Living Room | Rreach: 0.763 -> 0.771 | Helped |
Surprising findings
- Collision and floor-plan guidances conflict: collision guidance pushes objects apart while layout guidance pulls them inward, so tuning is needed to balance them.
- ATISS achieves better floor-plan violation and reachability scores than PhyScene in the bedroom setting, attributed to prioritizing floor plan constraints over collisions.
Limitations
- Restricted to limited room types (bedroom, dining, living room) and does not include small objects, limiting pick-and-place tasks.
- Collision is approximated with 3D bounding boxes rather than exact mesh collisions.
Experimental setup
Datasets: 3D-FRONT 3D-FUTURE GAPartNet
Benchmarks: 3D-FRONT
Metrics: FID KID SCA CKL Colobj Colscene Rout Rreach Rwalkable
Baselines: ATISS DiffuScene
Builds on: DiffuScene ATISS classifier guidance SceneDiffuser DDPM
Closest papers by meaning
- SceneWeaver: All-in-One 3D Scene Synthesis with an Extensible and Self-Reflective Agent (NeurIPS 2025 · similarity 0.89)
- InternScenes: A Large-scale Simulatable Indoor Scene Dataset with Realistic Layouts (NeurIPS 2025 · similarity 0.87)
- EmbodiedScan: A Holistic Multi-Modal 3D Perception Suite Towards Embodied AI (CVPR 2024 · similarity 0.87)
- Architect: Generating Vivid and Interactive 3D Scenes with Hierarchical 2D Inpainting (NeurIPS 2024 · similarity 0.87)
- PhyBlock: A Progressive Benchmark for Physical Understanding and Planning via 3D Block Assembly (NeurIPS 2025 · similarity 0.86)
- TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.