AI Paper Atlas

Research map / 3D vision / 3D generation

InfCoiL: Coordinated Planner-Controller Learning for Closed-Loop Physics-Based Human-Object Interaction

Yude Zou, Yifei Yao, Yiwei Hao, Hanqing Wang, Xing Gao

NeurIPS 2026 · 3D vision › 3D generation · approach: Hand-object interaction generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes InfCoiL, a coordinated planner-controller framework that closes the loop between a high-level HOI planner and a physics-based controller for text- and goal-conditioned human-object interaction.

Problem

Text- and goal-conditioned physics-based human-object interaction requires hierarchical planning and control, but existing planner-controller systems are open-loop and do not adapt to physical execution feedback.

Approach

Coordinates planner and controller learning in a closed loop so that the planner adapts to physical execution feedback from the controller, rather than issuing open-loop plans.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add closed-loop receding-horizon replanning (ablation)InfCoiL (frozen planner and controller) · OMOMO re-split held-out test (closed-loop task eval)SR@0.3 (%): 49.23 -> 56.09Improved
fine-tune the controller with PPO on closed-loop rollouts (ablation)InfCoiL · OMOMO re-split held-out test (closed-loop task eval)SR@0.3 (%): 56.09 -> 73.37Improved
fine-tune only the planner with SFPO against a frozen controller (ablation)InfCoiL · OMOMO re-split held-out test (closed-loop task eval)SR@0.3 (%): 56.09 -> 56.98No clear effect
add SFPO planner phase after controller fine-tuning (ablation)InfCoiL · OMOMO re-split held-out test (closed-loop task eval)SR@0.3 (%): 73.37 -> 77.24Improved
add a further controller phase after planner phase (ablation)InfCoiL · OMOMO re-split held-out test (closed-loop task eval)SR@0.3 (%): 77.24 -> 81.19Improved
continue controller-only training to 14k iterations instead of alternating (ablation)InfCoiL · Matched controller budgetSR@0.3 (%): 73.37 -> 71.99Hurt
replace critic-based FPO++ updates with SFPO (ablation)planner RL · Planner RL ablationSR@0.3 (%): 12.83 -> 77.24Improved
use isotropic noise (Flow-GRPO) instead of semantic channel-wise noise (ablation)planner RL · Planner RL ablationSR@0.3 (%): 77.24 -> 22.52Hurt
fine-tune planner by SFT instead of online RL (ablation)InfCoiL · Closed-loop executionSR@0.3 (%): 73.37 -> 71.75Hurt
fine-tune planner by SFT or SFPO under open-loop execution (ablation)InfCoiL · Open-loop executionSR@0.3 (%): 52.87 -> 54.40 (SFT), 53.35 (SFPO)No clear effect
add contact-aware state initialization (ablation)InfCoiL Teacher · Reference-tracking subset-2SR (%): 67.86 -> 92.86Improved
predict clean state x0 with x-space loss instead of velocity (ablation)flow-matching planner · OMOMO open-loop generationfoot sliding: 0.579 -> 0.350Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy