Research map / Video / Video generation
HOIAnimator: Generating Text-prompt Human-object Animations using Novel Perceptive Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HOIAnimator, a dual-branch diffusion model (human-centric and object-centric) with message passing and an interaction contact field that generates text-driven human-object interaction animations.
Problem
Generating realistic human-object interaction animations directly from text, where human and object motion must be consistent and contacts plausible.
Approach
Two coupled diffusion models (human and object) exchange features via Perceptive Message Passing (object-conditioned affine modulation plus dual-flow aggregation), and an Interaction Contact Field diffusion predicts text-conditioned human-object distance/penetration fields that are cross-attended into the animation generator.
Key results
- On BEHAVE, Top-3 R-precision improves from 0.734 (InterGen) to 0.781 and FID from 0.717 to 0.623.
- Vertex distance drops from 0.523 (InterGen) to 0.118, and penetration score rises to 0.643 versus 0.613 for InterGen.
- Ablation: removing PDM drops Top-3 precision to 0.696 and FID rises to 0.823; removing ICF raises FID to 0.714.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HOIAnimator instead of InterGen | Text-driven HOI animation generation · BEHAVE | R Precision Top 3: 0.734±0.005 -> 0.781±0.005 | Helped |
| Use HOIAnimator instead of InterGen | Text-driven HOI animation generation · BEHAVE | Vertex Distance: 0.523±0.005 -> 0.118±0.063 | Helped |
| Use HOIAnimator instead of InterGen | Text-driven HOI animation generation · BEHAVE | FID: 0.717±0.055 -> 0.623±0.063 | Helped |
| Use HOIAnimator instead of InterGen | Text-driven HOI animation generation · BEHAVE | Penetration: 0.613±0.001 -> 0.643±0.001 | Helped |
| Add ICF (ablation) | HOIAnimator · BEHAVE | Precision Top 3: 0.756±0.004 -> 0.781±0.005 | Helped |
| Add PMP (ablation) | HOIAnimator · BEHAVE | Precision Top 3: 0.723±0.006 -> 0.781±0.005 | Helped |
| Use dual PDM instead of a single diffusion model (ablation) | HOIAnimator · BEHAVE | Precision Top 3: 0.696±0.008 -> 0.781±0.005 | Helped |
Limitations
- Less adept at complex sequences of interactions
- Limited to scenes with a single object interacting at a time (multiple simultaneous objects not handled)
- Does not support non-rigid object animation such as water
Experimental setup
Datasets: BEHAVE InterCap
Benchmarks: BEHAVE
Metrics: R Precision (Top 1/2/3) FID MM Dist Diversity Vertex Distance Penetration score
Baselines: MotionCLIP T2M MDM MLD PriorMDM InterGen
Builds on: MDM InterGen PriorMDM CLIP
Closest papers by meaning
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.91)
- Auto-Regressive Diffusion for Generating 3D Human-Object Interactions (AAAI 2025 · similarity 0.90)
- CG-HOI: Contact-Guided 3D Human-Object Interaction Generation (CVPR 2024 · similarity 0.89)
- InteractDiffusion: Interaction Control in Text-to-Image Diffusion Models (CVPR 2024 · similarity 0.89)
- Enhanced Fine-Grained Motion Diffusion for Text-Driven Human Motion Synthesis (AAAI 2024 · similarity 0.89)
- AMD: Autoregressive Motion Diffusion (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.