Research map / 3D vision / 3D generation
InfCoiL: Coordinated Planner-Controller Learning for Closed-Loop Physics-Based Human-Object Interaction
Yude Zou, Yifei Yao, Yiwei Hao, Hanqing Wang, Xing Gao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InfCoiL, a coordinated planner-controller framework that closes the loop between a high-level HOI planner and a physics-based controller for text- and goal-conditioned human-object interaction.
Problem
Text- and goal-conditioned physics-based human-object interaction requires hierarchical planning and control, but existing planner-controller systems are open-loop and do not adapt to physical execution feedback.
Approach
Coordinates planner and controller learning in a closed loop so that the planner adapts to physical execution feedback from the controller, rather than issuing open-loop plans.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add closed-loop receding-horizon replanning (ablation) | InfCoiL (frozen planner and controller) · OMOMO re-split held-out test (closed-loop task eval) | SR@0.3 (%): 49.23 -> 56.09 | Improved |
| fine-tune the controller with PPO on closed-loop rollouts (ablation) | InfCoiL · OMOMO re-split held-out test (closed-loop task eval) | SR@0.3 (%): 56.09 -> 73.37 | Improved |
| fine-tune only the planner with SFPO against a frozen controller (ablation) | InfCoiL · OMOMO re-split held-out test (closed-loop task eval) | SR@0.3 (%): 56.09 -> 56.98 | No clear effect |
| add SFPO planner phase after controller fine-tuning (ablation) | InfCoiL · OMOMO re-split held-out test (closed-loop task eval) | SR@0.3 (%): 73.37 -> 77.24 | Improved |
| add a further controller phase after planner phase (ablation) | InfCoiL · OMOMO re-split held-out test (closed-loop task eval) | SR@0.3 (%): 77.24 -> 81.19 | Improved |
| continue controller-only training to 14k iterations instead of alternating (ablation) | InfCoiL · Matched controller budget | SR@0.3 (%): 73.37 -> 71.99 | Hurt |
| replace critic-based FPO++ updates with SFPO (ablation) | planner RL · Planner RL ablation | SR@0.3 (%): 12.83 -> 77.24 | Improved |
| use isotropic noise (Flow-GRPO) instead of semantic channel-wise noise (ablation) | planner RL · Planner RL ablation | SR@0.3 (%): 77.24 -> 22.52 | Hurt |
| fine-tune planner by SFT instead of online RL (ablation) | InfCoiL · Closed-loop execution | SR@0.3 (%): 73.37 -> 71.75 | Hurt |
| fine-tune planner by SFT or SFPO under open-loop execution (ablation) | InfCoiL · Open-loop execution | SR@0.3 (%): 52.87 -> 54.40 (SFT), 53.35 (SFPO) | No clear effect |
| add contact-aware state initialization (ablation) | InfCoiL Teacher · Reference-tracking subset-2 | SR (%): 67.86 -> 92.86 | Improved |
| predict clean state x0 with x-space loss instead of velocity (ablation) | flow-matching planner · OMOMO open-loop generation | foot sliding: 0.579 -> 0.350 | Improved |
Closest papers by meaning
- TokenHSI: Unified Synthesis of Physical Human-Scene Interactions through Task Tokenization (CVPR 2025 · similarity 0.88)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.88)
- CooHOI: Learning Cooperative Human-Object Interaction with Manipulated Object Dynamics (NeurIPS 2024 · similarity 0.87)
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.86)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.86)
- InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
