Research map / Detection and segmentation / Open-set and few-shot detection
Learning Human-Object Interaction as Groups
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GroupHOI, a DETR-based HOI detector that groups entities by geometric proximity (learned estimator over box distance and IoU) and interactions by semantic similarity, reaching 36.70 mAP on HICO-DET and state-of-the-art on NVI-DET.
Problem
Human-object interaction detection typically models only pairwise or fully-connected relations among entities, ignoring collective group structure in multi-person, multi-object scenes.
Approach
Revisit relation modeling from a group view: build per-entity geometric groups via a learnable proximity estimator on center distance and IoU with local self-attention, and per-query semantic groups via top-k cosine similarity with max-pooled local context added to interaction queries.
Key results
- GroupHOI with ResNet-50 and CLIP ViT-B/16 reaches 36.70 mAP Full on HICO-DET, +0.84 over Pose-Aware (35.86).
- With BLIP2 ViT-L, GroupHOI reaches 40.53 Full mAP on HICO-DET, above UniHOI (40.06).
- On NVI test, GroupHOI achieves 75.21 AR versus 74.67 for NVI-DEHR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add geometric groups (ablation) | GroupHOI baseline without groups · HICO-DET test, Default Full | mAP: 35.72 -> 36.39 | Helped |
| Add semantic groups (ablation) | GroupHOI baseline without groups · HICO-DET test, Default Full | mAP: 35.72 -> 35.93 | Helped |
| Add geometric and semantic groups together (ablation) | GroupHOI baseline without groups · HICO-DET test, Default Full | mAP: 35.72 -> 36.70 | Helped |
| Use homogeneous rather than heterogeneous geometric groups (ablation) | GroupHOI · HICO-DET test, Default Full | mAP: 36.70 -> 36.23 | Hurt |
| Set geometric group size to 2, 3, or 5 instead of 4 (ablation) | GroupHOI · HICO-DET test, Default Full | mAP: 36.70 -> 36.23; 36.70 -> 36.34; 36.70 -> 36.45 | Hurt |
| Set semantic group size to 1, 3, or 4 instead of 2 (ablation) | GroupHOI · HICO-DET test, Default Full | mAP: 36.70 -> 36.07; 36.70 -> 36.18; 36.70 -> 35.92 | Hurt |
| Increase geometric group layers from 1 to 2, 3, or 4 (ablation) | GroupHOI · HICO-DET test, Default Full | mAP: 36.70 -> 36.47; 36.70 -> 35.64; 36.70 -> 36.17 | Mixed |
| Use 1, 2, or 4 semantic group layers instead of 3 (ablation) | GroupHOI · HICO-DET test, Default Full | mAP: 36.70 -> 36.13; 36.70 -> 36.53; 36.70 -> 36.13 | Hurt |
| Use IoU alone or center distance alone rather than both (ablation) | GroupHOI geometric proximity estimator · HICO-DET test, Default Full | mAP: 36.70 -> 36.02; 36.70 -> 36.14 | Hurt |
| Add global image features to IoU and center distance (ablation) | GroupHOI geometric proximity estimator · HICO-DET test, Default Full | mAP: 36.70 -> 36.21 | Hurt |
| Use GroupHOI with CLIP ViT-B/16 instead of Pose-Aware | HICO-DET test · Default Full | mAP: 35.86 -> 36.70 (+0.84) | Helped |
| Use GroupHOI instead of NVI-DEHR | NVI · val | AR: 71.20 -> 73.19 | Helped |
Surprising findings
- On V-COCO, GroupHOI is slightly inferior to STIP, VIL, and CQL in role AP, despite gains on HICO-DET.
- Adding global image features to the geometric proximity estimator slightly degrades performance.
Limitations
- Grouping is applied only to interaction reasoning, not the object detection branch.
- Trained with limited label diversity, which hampers generalization to in-the-wild scenes.
- Lower V-COCO performance than some prior methods.
Experimental setup
Datasets: HICO-DET V-COCO NVI MS-COCO
Benchmarks: HICO-DET V-COCO NVI
Metrics: mAP Recall@K (mR@K) AR FPS FLOPs Params
Baselines: QPIC CDN GEN-VLKT HOICLIP Pose-Aware UniHOI CMMP STIP VIL CQL NVI-DEHR ViPLO
Builds on: DETR HOICLIP GEN-VLKT CLIP
Closest papers by meaning
- ContextHOI: Spatial Context Learning for Human-Object Interaction Detection (AAAI 2025 · similarity 0.91)
- HORP: Human-Object Relation Priors Guided HOI Detection (CVPR 2025 · similarity 0.90)
- Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models (NeurIPS 2024 · similarity 0.90)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.90)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.89)
- Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.