Research map / Detection and segmentation / Open-set and few-shot detection
Bilateral Collaboration with Large Vision-Language Models for Open Vocabulary Human-Object Interaction Detection
Yupeng Hu, Changxing Ding, Chang Sun, Shaoli Huang, Xiangmin Xu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add attention bias guidance (ABG) from the HOI detector to the VLM (ablation) | BC-HOI with early fusion · HICO-DET closed (Default) | mAP (Full): 39.34 -> 42.60 | Improved |
| add LLM-based supervision guidance (LSG) (ablation) | BC-HOI with early fusion · HICO-DET closed (Default) | mAP (Full): 39.34 -> 42.41 | Improved |
| add early fusion of CNN and BLIP-2 feature maps (ablation) | late-fusion baseline · HICO-DET closed (Default) | mAP (Full): 38.91 -> 39.34 | Improved |
| replace detector-provided adaptive attention bias with plain learnable bias (ablation) | BC-HOI · HICO-DET NF-UC | mAP (Full): 36.40 -> 32.60 | Hurt |
| use token-level LLM supervision rather than caption-level holistic supervision (ablation) | LSG · HICO-DET NF-UC | mAP (Full): 35.56 -> 36.40 | Improved |
| take attention bias from both detection and interaction decoders (averaged) instead of interaction decoder only (ablation) | ABG · HICO-DET NF-UC | mAP (Full): 36.40 -> 35.62 | Hurt |
| apply attention bias to both ViT encoder and Q-Former (ablation) | ABG · HICO-DET NF-UC | mAP (Full): 35.45 (ViT only) -> 36.40 | Improved |
| predict all caption tokens with larger weights on nouns and verbs (ablation) | LSG · HICO-DET NF-UC | mAP (Full): 35.74 (nouns and verbs only) -> 36.14 (equal weights) -> 36.40 | Improved |
| use QwenVL-7B instead of GPT-4o for captions (ablation) | LSG · HICO-DET NF-UC | mAP (Full): 36.40 -> 36.24 | No clear effect |
| BC-HOI versus UniHOI on UO unseen categories | BC-HOI · HICO-DET UO | mAP (Unseen): 19.72 -> 19.94 | No clear effect |
| add VLM interaction via ABG/LSG versus UniHOI | BC-HOI · Inference cost | time per image: 82.3ms -> 99.7ms | Hurt |
Closest papers by meaning
- Streamlined Open-Vocabulary Human-Object Interaction Detection (CVPR 2026 · similarity 0.94)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.94)
- Discovering Syntactic Interaction Clues for Human-Object Interaction Detection (CVPR 2024 · similarity 0.92)
- Open-World Human-Object Interaction Detection via Multi-modal Prompts (CVPR 2024 · similarity 0.91)
- SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detection (CVPR 2025 · similarity 0.90)
- Toward Open-Set Human Object Interaction Detection (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
