Research map / Multi-agent systems and games / Human-AI and goal-conditioned
CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CooHOI, a two-phase framework that trains a single humanoid to carry objects with AMP, then transfers the skill to multi-humanoid cooperative transport via MAPPO, using shared object dynamics as implicit communication.
Problem
Multi-humanoid cooperative transport of large, heavy objects is hard because mocap data for multi-character HOI is scarce and multi-agent RL from scratch converges poorly due to the enlarged action space.
Approach
Train a single-agent carrying skill with AMP using object bounding-box dynamics in the observation, then transfer it to two or more agents by observing the bounding boxes at the object's ends and training with CTDE (MAPPO) and parameter sharing, so object dynamics act as implicit communication.
Key results
- Single-agent box carrying reaches 96.48% success rate with 6.9 cm precision, versus InterPhys 94.3% and 8.3 cm.
- Two-agent box carrying reaches 89.54% success with 3.86 cm precision, while training from scratch achieves 0%.
- Two-agent sofa carrying achieves 84.17% success with 10.12 cm precision.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize two-agent training from the single-agent policy and fine-tune it, rather than training from scratch (ablation) | CooHOI · Box carrying, two agents | Success Rate (%): 0 -> 89.54 | Helped |
| Add weight augmentation (ablation) | CooHOI single-agent box carrying · Box carrying, single agent | Success Rate (%): 96.48 -> 93.98 | Mixed |
| Add weight augmentation (ablation) | CooHOI single-agent box carrying · Box carrying, single agent | Precision (cm): 6.9 -> 4.8 | Helped |
| Add a stand point that directs the agent toward an appropriate object face (ablation) | CooHOI · Box carrying, two agents | task completion | Helped |
| Include object dynamics in each agent's observation (ablation) | CooHOI · Box carrying, two agents | task completion | Helped |
| Train the single-agent policy with backward-walking motion and a reverse-walk reward (ablation) | CooHOI · Box carrying, two agents | task completion | Helped |
| Initialize the cooperative policy from the single-agent policy (ablation) | CooHOI · Box carrying, two agents | task completion | Helped |
| Increase observation-noise standard deviation from 0 to 4 (ablation) | CooHOI single-agent box carrying · Box carrying, single agent | Success Rate (%): 96.85 -> 48.48 | Hurt |
| Increase observation-noise standard deviation from 0 to 4 (ablation) | CooHOI two-agent box carrying · Box carrying, two agents | Success Rate (%): 90.33 -> 79.93 | Hurt |
| Increase box weight to 20 kg (ablation) | CooHOI single-agent box carrying · Box carrying, single agent | accuracy: 30% | Hurt |
| Restrict object width to 1x and scale length and height to 1.5x (ablation) | CooHOI single-agent box carrying · Box carrying, single agent | Success rate: 97.8% | Helped |
Surprising findings
- Multi-agent training from scratch fails completely (0% success), even with extended training (4x steps).
- Without reverse-walk data, two agents deadlock by pushing the box against each other.
Limitations
- Humanoids lack dexterous hands, limiting manipulation of slippery or precise objects.
- Object represented only by bounding box, limiting generalization across object shapes.
- Only evaluated in simulation with no real-world deployment.
Experimental setup
Datasets: AMASS 26 reference motion clips (walk, pick up, carry, put down, sidewalk, reverse carry) daily-life object meshes (nearly 40 objects)
Benchmarks: box carrying (single and two agent, IsaacGym simulation) daily-life objects (Table, Armchair, HighStool, Sofa)
Metrics: success rate precision (cm)
Baselines: InterPhys From Scratch multi-agent training
Builds on: AMP MAPPO InterPhys
Closest papers by meaning
- Habitat 3.0: A Co-Habitat for Humans, Avatars, and Robots (ICLR 2024 · similarity 0.86)
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.86)
- VIKI‑R: Coordinating Embodied Multi-Agent Cooperation via Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
- CaPo: Cooperative Plan Optimization for Efficient Embodied Multi-Agent Cooperation (ICLR 2025 · similarity 0.85)
- TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization (CVPR 2025 · similarity 0.85)
- LLawCo: Learning Laws of Cooperation for Modeling Embodied Multi-Agent Behavior (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.