Research map / Detection and segmentation / Open-set and few-shot detection
ContextHOI: Spatial Context Learning for Human-Object Interaction Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ContextHOI, a dual-branch DETR-based HOI detector that learns spatial context with spatially contrastive constraints and CLIP-guided context exploration, and introduces a HICO-DET (ambiguous) benchmark of occluded/blurred instances.
Problem
Existing transformer-based HOI detectors focus on instance-centric features and recognize interactions poorly when the foreground person or object is blurred, occluded, or unseen, because they capture little spatial context.
Approach
A parallel context branch with its own context extractor and queries learns background regions without hand-crafted background labels, supervised by feature-, region-, and instance-level contrastive constraints (with a dynamic distance weight on the GIoU loss) that push context away from instance regions, plus semantic guidance distilled from CLIP text embeddings.
Key results
- ContextHOI (ResNet50) reaches 41.82 full mAP on HICO-DET default, the best among compared methods listed, with 43.91 rare mAP.
- On HICO-DET (ambiguous), ContextHOI reaches 46.99 full mAP, +5.17 over QPIC, and 60.57 rare mAP, +16.66 over QPIC.
- On V-COCO, ContextHOI reaches 66.1 AP role (R50) and 67.3 (R101).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the context branch to the base model (ablation) | ContextHOI · HICO-DET Default full | mAP: 34.69 -> 35.81 | Helped |
| add spatially contrastive constraints to the context branch (ablation) | ContextHOI · HICO-DET Default full | mAP: 35.81 -> 36.75 | Helped |
| add semantic-guided context explorers to the context branch (ablation) | ContextHOI · HICO-DET Default full | mAP: 35.81 -> 38.91 | Helped |
| combine spatial constraints and semantic-guided explorers (ablation) | ContextHOI · HICO-DET Default full | mAP: 38.91 -> 41.82 | Helped |
| add feature-level spatial constraint (ablation) | ContextHOI · HICO-DET Default full | mAP: 38.91 -> 40.93 | Helped |
| add region-level spatial constraint (ablation) | ContextHOI · HICO-DET Default full | mAP: 38.91 -> 41.41 | Helped |
| add instance-level constraint without dynamic distance weighting (ablation) | ContextHOI · HICO-DET Default full | mAP: 38.91 -> 40.59 | Helped |
| add dynamic distance weight to the instance-level constraint (ablation) | ContextHOI · HICO-DET Default full | mAP: 40.59 -> 41.46 | Helped |
| use CLIP-L/14 instead of no VLM prior (ablation) | ContextHOI · HICO-DET Default full | mAP: 36.75 -> 41.82 | Helped |
| replace EVA-01 prior with CLIP-L/14 prior (ablation) | ContextHOI · HICO-DET Default full | mAP: 40.28 -> 41.82 | Helped |
| add VLM visual features to the context aggregator (ablation) | ContextHOI · HICO-DET Default full | mAP: 39.92 -> 41.82 | Helped |
| use ContextHOI rather than QPIC on ambiguous scenes | HOI detection · HICO-DET ambiguous full | mAP: +37.74 | Helped |
Surprising findings
- Spatial constraints alone push the context extractor toward image margins, producing noise; semantic guidance alone makes it duplicate the instance decoder.
Limitations
- Context regions can drift toward image edges without the dynamic distance weight.
- Ambiguous benchmark is a 659-image subset selected by volunteers, which is small.
- Inference cost is higher than HOICLIP (13.37 vs 16.47 FPS).
Experimental setup
Datasets: HICO-DET V-COCO MS-COCO
Benchmarks: HICO-DET V-COCO HICO-DET (ambiguous)
Metrics: mAP
Baselines: QPIC HOICLIP UPT ADA-CM BCOM UniHOI RLIPv2 PViC
Builds on: DETR QPIC CLIP UPT CDN
Closest papers by meaning
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.91)
- Bilateral Adaptation for Human-Object Interaction Detection with Occlusion-Robustness (CVPR 2024 · similarity 0.91)
- Discovering Syntactic Interaction Clues for Human-Object Interaction Detection (CVPR 2024 · similarity 0.88)
- Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.88)
- Locality-Aware Zero-Shot Human-Object Interaction Detection (CVPR 2025 · similarity 0.88)
- Interacted Object Grounding in Spatio-Temporal Human-Object Interactions (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.