Research map / Detection and segmentation / Open-set and few-shot detection
HORP: Human-Object Relation Priors Guided HOI Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HORP, a two-stage HOI detector that adds text-encoded human-object 3D location priors and gaze-area priors to visual query features to better recognize the no-interaction category.
Problem
HOI detection performance is unsatisfactory, largely because the no-interaction category has very low accuracy and is confused with direct and indirect interactions.
Approach
Encode a pairwise 3D location prior (2D IoU plus relative scale-based depth) and a gaze-area prior (fraction of object inside the predicted gaze cone) as text via a frozen CLIP text encoder, then concatenate these text features with visual pair features as decoder queries.
Key results
- HORP with ResNet-50+CLIP reaches 38.61 mAP Full on HICO-DET Default, +3.51 over the 35.10 baseline, and no-interaction AP rises from 16.32 to 23.11.
- HORP with Swin-L+CLIP reaches 47.53 mAP Full on HICO-DET Default, surpassing PAFR (46.01) by 1.53 pp.
- On V-COCO, HORP Swin-L achieves 51.24 (scenario 1), the best among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add 3D location and gaze area priors | HORP with Swin-L+CLIP · HICO-DET Default, Full | mAP: +1.53 pp | Helped |
| Add 3D location and gaze area priors | HORP with Swin-L+CLIP · V-COCO Scenario 1 | mAP: 68.9 vs. 67.9 | Helped |
| Add gaze area prior only (ablation) | HORP · HICO-DET Default, Full | mAP: 35.10 -> 36.85 | Helped |
| Add 3D location prior only (ablation) | HORP · HICO-DET Default, Full | mAP: 35.10 -> 37.24 | Helped |
| Add both human-object relation priors (ablation) | HORP · HICO-DET Default, Full | mAP: 35.10 -> 38.61 | Helped |
| Add gaze area prior only (ablation) | HORP · HICO-DET no-interaction category | AP: 16.32 -> 19.65 | Helped |
| Add 3D location prior only (ablation) | HORP · HICO-DET no-interaction category | AP: 16.32 -> 20.16 | Helped |
| Add both human-object relation priors (ablation) | HORP · HICO-DET no-interaction category | AP: 16.32 -> 23.11 | Helped |
| Use 2D spatial information only for the location prior (ablation) | HORP · HICO-DET Default, Full | mAP: 36.85 -> 37.22 | Helped |
| Use depth information only for the location prior (ablation) | HORP · HICO-DET Default, Full | mAP: 36.85 -> 37.45 | Helped |
| Combine 2D spatial and depth information in the location prior (ablation) | HORP · HICO-DET Default, Full | mAP: 36.85 -> 38.61 | Helped |
| Use weighted sum instead of concatenation to construct queries (ablation) | HORP · HICO-DET Default, Full | mAP: 38.61 vs. 37.75 | No clear effect |
Surprising findings
- No-interaction accuracy is about 40 pp lower than with-interaction categories, which the authors identify as the main bottleneck for overall HOI performance.
Limitations
- Depth is only roughly estimated from pixel counts per category rather than from a depth model.
- Gaze prior depends on an off-the-shelf gaze cone predictor that is fixed and not trained jointly.
Experimental setup
Datasets: HICO-DET V-COCO
Benchmarks: HICO-DET V-COCO
Metrics: mAP AP for no-interaction
Baselines: PVIC UPT CDN STIP AGER ViPLO GEN-VLKT ADA-CM SICHOI DiffHOI PAFR RLIPv2
Builds on: UPT PVIC DETR Focal Loss
Closest papers by meaning
- Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models (NeurIPS 2023 · similarity 0.93)
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.90)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.90)
- Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.89)
- HOI-M^3: Capture Multiple Humans and Objects Interaction within Contextual Environment (CVPR 2024 · similarity 0.89)
- HOIMamba: Efficient Mamba-based Disentangled Progressive Learning for HOI Detection (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.