Research map / Vision-language and multimodal / Human and embodied modeling
Inter-X: Towards Versatile Human-Human Interaction Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Inter-X, a MoCap human-human interaction dataset (~11K sequences, 8.1M frames, 34K part-level texts, hand gestures, interaction order, relationship and personality labels) and a 4-category benchmark of generation and recognition tasks.
Problem
Existing human-human interaction datasets lack accurate body motion, hand gestures, fine-grained part-level text, interaction order asymmetry, and relationship/personality annotations, limiting versatile interaction perception and generation.
Approach
Capture human-human interactions with an optical MoCap system for body and inertial gloves for fingers, then annotate with part-level text, action categories, actor/reactor order, and relationship/Big Five personality labels to support a unified benchmark of four task families.
Key results
- InterGen achieves the best text-conditioned generation results on most metrics (R Precision Top-1 0.207, FID 5.207).
- MS-G3D reaches the best skeleton-based interaction recognition Top-1 accuracy of 83.30%.
- AGRoL gives the best human reaction generation results (FID 4.386, accuracy 0.925).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use InterGen instead of the other evaluated text-conditioned generation models | Text-conditioned interaction generation · Inter-X test set | FID: 5.207±0.2160 | Helped |
| Use TEMOS instead of the other evaluated text-conditioned generation models | Text-conditioned interaction generation · Inter-X test set | MM Dist: 6.867±0.013 | Helped |
| Use MDM(GRU) instead of the other evaluated text-conditioned generation models | Text-conditioned interaction generation · Inter-X test set | Diversity: 7.003±0.134 | Mixed |
| Use InterGen instead of the other evaluated text-conditioned generation models | Text-conditioned interaction generation · Inter-X test set | MModality: 3.686±0.052 | Helped |
| Use Actformer instead of the other evaluated action-conditioned generation models | Action-conditioned interaction generation · Inter-X | FID: 8.067±0.653 | Helped |
| Use Actformer instead of the other evaluated action-conditioned generation models | Action-conditioned interaction generation · Inter-X | Action recognition accuracy: 0.945±0.0007 | Helped |
| Use MDM instead of the other evaluated action-conditioned generation models | Action-conditioned interaction generation · Inter-X | Multimodality: 22.042±0.153 | Helped |
| Use AGRoL instead of the other evaluated reaction-generation models | Human reaction generation · Inter-X | FID: 4.386±0.2186 | Helped |
| Use AGRoL instead of the other evaluated reaction-generation models | Human reaction generation · Inter-X | Action recognition accuracy: 0.925±0.0001 | Helped |
| Use MS-G3D instead of the other evaluated skeleton recognition models | Human interaction recognition · Inter-X | Top-1 accuracy: 83.30% | Helped |
Surprising findings
- ComMDM achieves the worst R Precision scores, which the authors attribute to its need for extra pre-training.
- Recognition accuracy of 83.30% for MS-G3D is described as not satisfactory, indicating dexterous hand gestures pose new challenges.
Limitations
- No facial expressions, since the indoor MoCap setup used non-professional actors.
- Only atomic interaction sequences rather than long, multi-transition interactions.
- Action-conditioned and recognition baselines are re-implemented and evaluated on a single dataset without cross-dataset checks.
Experimental setup
Datasets: Inter-X
Benchmarks: Inter-X
Metrics: FID R Precision MM Dist Diversity MModality action recognition accuracy Top-1 accuracy Top-5 accuracy
Baselines: TEMOS T2M MDM MDM-GRU ComMDM InterGen Action2Motion ACTOR Actformer RAIG AGRoL ST-GCN
Builds on: InterGen SMPL-X MDM ST-GCN
Closest papers by meaning
- Learning to Generate Human-Human-Object Interactions from Textual Descriptions (NeurIPS 2025 · similarity 0.87)
- InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.87)
- InterDreamer: Zero-Shot Text to 3D Dynamic Human-Object Interaction (NeurIPS 2024 · similarity 0.87)
- Expressive Forecasting of 3D Whole-Body Human Motions (AAAI 2024 · similarity 0.87)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.86)
- Reconstructing Close Human Interaction with Appearance and Proxemics Reasoning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.