Research map / Video / Video generation
InterControl: Zero-shot Human Interaction Generation by Controlling Every Joint
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates multi-person interactions zero-shot by training a single-person motion diffusion model (MDM) with a Motion ControlNet for every-joint spatial control plus L-BFGS inverse-kinematics guidance, with GPT-4 converting interaction text into joint-contact pairs.
Problem
Text-conditioned motion diffusion models target single characters and cannot generate multi-human interactions for arbitrary numbers of people; existing interaction methods are tied to fixed character counts and need multi-person data.
Approach
Treat human-human interactions as joint-joint contact or separation pairs and realize them as precise every-joint spatial control on a single-person MDM, using a trainable Motion ControlNet plus test-time IK guidance (L-BFGS) with an off-the-shelf GPT-4 planner to produce contact plans.
Key results
- On HumanML3D root control, InterControl reaches FID 0.159 vs 0.218 for OmniControl and 0.212 for GMD, with location error 0.0004 at 50 cm threshold.
- In a user study with 134 participants, InterControl was preferred over PriorMDM for two-person interactions 81.2% of the time.
- On 100 InterHuman-derived interaction prompts (989 valid plans), InterControl achieves far lower spatial errors than PriorMDM (avg. err. 0.0084 m vs 0.6723 m).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Motion ControlNet (ablation) | InterControl · HumanML3D, random one joint | FID: 0.965 -> 0.178 | Helped |
| use c_final instead of original c (ablation) | InterControl Motion ControlNet · HumanML3D, random one joint | FID: 0.227 -> 0.178 | Helped |
| add IK guidance (ablation) | InterControl · HumanML3D, random one joint | Trajectory error (50 cm): 0.8569 -> 0.0403 | Helped |
| apply IK guidance to the posterior mean instead of x0 (ablation) | InterControl · HumanML3D, random one joint | FID: 0.211 -> 0.178 | Helped |
| use L-BFGS instead of first-order gradient (ablation) | InterControl IK guidance · HumanML3D, random one joint | FID: 0.198 -> 0.178 | Helped |
| use control sparsity 0.25 instead of the full-method setting (ablation) | InterControl · HumanML3D, random one joint | FID: 0.178 -> 0.248 | Hurt |
| use control sparsity 0.025 instead of the full-method setting (ablation) | InterControl · HumanML3D, random one joint | FID: 0.178 -> 0.255 | Hurt |
| use IK guidance instead of no spatial guidance (ablation) | InterControl · HumanML3D, random one joint | Trajectory error (50 cm): 0.8569 -> 0.0403 | Helped |
| use InterControl rather than PriorMDM | Two-person interaction generation · HumanML3D, interaction control (20 cm) | Trajectory error: 0.6931 -> 0.0082 | Helped |
| use InterControl rather than OmniControl | HumanML3D spatial control · Root control | FID: 0.218 -> 0.159 | Helped |
| use InterControl rather than OmniControl | HumanML3D spatial control · Random one joint | FID: 0.310 -> 0.178 | Helped |
| use InterControl rather than PriorMDM | Two-person interaction generation user study · User preference | Preference: 18.8% -> 81.2% | Helped |
Surprising findings
- Controlling joints with L-BFGS IK guidance alone (no ControlNet) gives low spatial error but severely damages FID (0.965), showing ControlNet is needed for motion quality.
Limitations
- Not trained on multi-person data, so interaction definitions are limited to distance or orientation constraints.
- Interaction plausibility depends on the LLM planner's contact plans; manual correction of LLM outputs was needed.
- Inference is slower than single-pass generation (about 80 s per sample on A100) due to IK guidance.
Experimental setup
Datasets: HumanML3D KIT-ML InterHuman
Benchmarks: HumanML3D KIT-ML
Metrics: FID R-precision diversity foot skating ratio trajectory error location error average error user preference
Baselines: PriorMDM GMD OmniControl MDM T2M-GPT MotionGPT MLD MotionDiffuse
Builds on: MDM ControlNet classifier guidance GMD PriorMDM
Closest papers by meaning
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.90)
- InterMimic: Towards Universal Whole-Body Control for Physics-Based Human-Object Interactions (CVPR 2025 · similarity 0.90)
- TIMotion: Temporal and Interactive Framework for Efficient Human-Human Motion Generation (CVPR 2025 · similarity 0.89)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.88)
- AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation (ICLR 2025 · similarity 0.88)
- Ready-to-React: Online Reaction Policy for Two-Character Interaction Generation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.