Research map / 3D vision / 3D generation
Sitcom-Crafter: A Plot-Driven Human Motion Generation System in 3D Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Sitcom-Crafter, a unified plot-driven system that generates locomotion, human-scene interaction, and scene-aware human-human interaction motions in 3D scenes, using a self-supervised synthetic SDF condition to reduce collisions.
Problem
Existing human motion generation methods each cover a single motion type (locomotion, human-scene interaction, or human-human interaction), and human-human interaction generators ignore scene geometry, causing collisions when composed in 3D scenes.
Approach
Synthesize binary SDF points from random plane-based virtual objects around the walkable region of existing human-human interaction data, use them as scene conditions for an InterGen-style generator, and combine with GAMMA/DIMOS and LLM-based plot parsing plus several augmentation modules into one system.
Key results
- On Replica full-system comparison, the proposed module achieves HSP 5.7529 vs 10.5532 (ComMDM) and 9.6035 (InterGen), and HHP 0.1687 vs 0.2712 and 0.1774.
- On InterHuman human-human interaction, HSP drops to 1.6950 vs 6.2802 (ComMDM) and 6.6408 (InterGen), with HHP 0.0742 vs 0.1336 and 0.0989.
- Foot sliding is 0 for all compared systems and the proposed method attains FP 0.0189 on Replica, versus 0.0215 (ComMDM) and 0.0242 (InterGen).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| scene-aware HHI module with synthetic SDF conditions (vs. InterGen and ComMDM) in the full system | human-human interaction module · Replica full system | HSP (human-scene penetration): 5.7529 vs 9.6035 (InterGen), 10.5532 (ComMDM) | Helped |
| scene-aware HHI module with synthetic SDF conditions (vs. InterGen and ComMDM) | human-human interaction module · Replica full system | HHP (human-human perturbation): 0.1687 vs 0.1774 (InterGen), 0.2712 (ComMDM) | Helped |
| scene-aware HHI module with synthetic SDF conditions (vs. InterGen) | human-human interaction module · InterHuman | HSP: 1.6950 vs 6.6408 (InterGen) | Helped |
| scene-aware HHI module with synthetic SDF conditions (vs. InterGen) | human-human interaction module · InterHuman | HHP: 0.0742 vs 0.0989 (InterGen) | Helped |
| SDF condition removed from the HHI generator (ablation) | human-human interaction module · Replica visual comparison | penetration (qualitative) | Hurt |
| new canonicalization (pelvis as local origin) vs InterGen canonicalization | human-human interaction module · InterHuman | generation quality (qualitative/bias analysis) | Helped |
| last-frame marker condition added to the denoiser | human-human interaction module · qualitative | motion collapse of two characters | Helped |
| Random plane-based height assignment for synthetic SDF objects (replacing random point-based heights) | synthetic SDF scene construction · qualitative | motion distortion during inference | Helped |
Limitations
- Synthetic SDF objects have only flat surfaces parallel to the floor; sloped surfaces are left to future work.
- Evaluation relies on physical-compliance metrics on 11 Replica scenes and no comprehensive open-source baseline exists for comparison, so baselines are constructed by the authors.
Experimental setup
Datasets: InterHuman Inter-X Replica
Benchmarks: Replica (11 indoor scenes, 110 plot-driven samples) InterHuman
Metrics: FS FP HSP HHP FID R-Precision Diversity MModality MM Dist
Baselines: InterGen ComMDM Baseline without human-human interaction
Builds on: InterGen GAMMA DIMOS MDM COINS
Closest papers by meaning
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint (NeurIPS 2024 · similarity 0.87)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.87)
- TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation (CVPR 2025 · similarity 0.86)
- HumanVid: Demystifying Training Data for Camera-controllable Human Image Animation (NeurIPS 2024 · similarity 0.86)
- SemGeoMo: Dynamic Contextual Human Motion Generation with Semantic and Geometric Guidance (CVPR 2025 · similarity 0.86)
- Generating Human Motion in 3D Scenes from Text Descriptions (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.