Research map / 3D vision / 3D generation
Digital Life Project: Autonomous 3D Characters with Social Intelligence
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Digital Life Project pairs an LLM-driven psychology-based "brain" (SocioMind) with a text-driven interactive motion synthesis "body" (MoMat-MoGen) to build autonomous 3D characters with social interaction and evolving relationships.
Problem
Building autonomous 3D characters that engage in social interaction with articulated body motion and show long-term social intelligence, which prior agent work (2D sandbox or limited 3D actions) does not address.
Approach
Uses text as the universal medium between an LLM "brain" and a motion "body": SocioMind prompts an LLM with IPIP-derived persona instructions, hierarchical psychological reflection and topic-proposal planning, while MoMat-MoGen combines text-based motion matching as a prior with a dual-path attention diffusion model (DSMA) plus contact-loss guidance for two-person motion.
Key results
- On InterHuman, MoMat-MoGen reaches FID 5.674 and Diversity 8.021 (vs. InterGen FID 5.918 in the best prior listed), with R Precision Top-1 0.449 comparable to ReMoDiffuse 0.442.
- In a 30-person user study, SocioMind + MoMat-MoGen scored 6.88 overall vs. 4.78 for SocioMind + InterGen and 6.07 for Generative Agents + MoMat-MoGen.
- Human evaluators (47) rated SocioMind higher than Generative Agents on controllability accuracy for central belief, motivation, personality, and relationship, and on plot and psychological-state consistency.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MoMat-MoGen instead of ReMoDiffuse | Interactive motion synthesis · InterHuman test | FID: 6.366±.102 -> 5.674±.085 | Helped |
| Use MoMat-MoGen instead of ReMoDiffuse | Interactive motion synthesis · InterHuman test | Diversity: 7.956±.030 -> 8.021±.035 | Helped |
| Use SocioMind + MoMat-MoGen instead of Generative Agents + InterGen | Integrated character pipeline · User study | Overall: 4.93 -> 6.88 | Helped |
| Add persona instructions (ablation) | SocioMind · Long-term social evolution | Consistency | Helped |
| Add psychological reflection (ablation) | SocioMind · Long-term social evolution | Psychological state consistency | Helped |
| Add topic proposal (ablation) | SocioMind · Long-term social evolution | Plot consistency | Helped |
Limitations
- Evaluation of social intelligence relies on human evaluators with a small number of episodes and scenarios.
- Motion generation relies on a small motion database for matching and the DLP-MoCap results are only in the supplementary material.
Experimental setup
Datasets: InterHuman DLP-MoCap IPIP
Benchmarks: InterHuman
Metrics: R Precision FID MM Dist Diversity MultiModality controllability accuracy consistency score user study script/motion/overall score
Baselines: ReMoDiffuse MotionDiffuse InterGen TEMOS T2M MDM ComMDM Generative Agents
Builds on: Generative Agents ReMoDiffuse Story-to-Motion Text-based Motion Matching IPIP
Closest papers by meaning
- SOLAMI: Social Vision-Language-Action Modeling for Immersive Interaction with 3D Autonomous Characters (CVPR 2025 · similarity 0.87)
- Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation (ICLR 2025 · similarity 0.86)
- The Language of Motion: Unifying Verbal and Non-verbal Language of 3D Human Motion (CVPR 2025 · similarity 0.85)
- Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots (ICLR 2024 · similarity 0.85)
- Unbounded: A Generative Infinite Game of Character Life Simulation (ICLR 2025 · similarity 0.84)
- Human Simulacra: Benchmarking the Personification of Large Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.