Research map / Detection and segmentation / Open-set and few-shot detection
SGC-Net: Stratified Granular Comparison Network for Open-Vocabulary HOI Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGC-Net, which uses multi-granularity visual features from intermediate CLIP layers and hierarchical LLM-generated text descriptions compared across groups to improve open-vocabulary HOI detection.
Problem
Open-vocabulary HOI detection suffers from feature granularity deficiency (only last-layer CLIP features used for text alignment) and semantic similarity confusion among interaction categories with similar semantics.
Approach
Granularity Sensing Alignment (GSA) aligns LLM-generated multi-granularity text descriptions with visual features from multiple CLIP layers, and Hierarchical Group Comparison (HGC) groups semantically similar interactions and compares them hierarchically using LLM-generated category and part-level descriptions to separate confusable categories.
Key results
- SGC-Net outperforms prior OV-HOI methods on SWIG-HOI and HICO-DET unseen-category settings (paper reports improvements over CLIP-based baselines in mAP).
- Ablation shows multi-layer features plus hierarchical comparison give gains; grouping with a small group size (N=6) balances construction time and accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add GSA to the baseline (ablation) | SGC-Net · SWIG-HOI | mAP: 11.45 -> 16.49 | Helped |
| Add HGC to the baseline (ablation) | SGC-Net · SWIG-HOI | mAP: 11.45 -> 14.81 | Helped |
| Combine GSA and HGC (ablation) | SGC-Net · SWIG-HOI Unseen | mAP: 10.69 -> 12.46 | Helped |
| Use layer split {6-8, 9-11, 12} rather than {4-6, 7-9, 10-12} (ablation) | SGC-Net · SWIG-HOI Full | mAP: 16.35 -> 17.20 | Helped |
| Increase the number of blocks from 1 to 3 (ablation) | SGC-Net · SWIG-HOI Unseen | mAP: 10.09 -> 12.46 | Helped |
| Reduce the number of blocks from 3 to 2 (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 14.78 | Hurt |
| Use 4 blocks instead of 3 (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 16.53 | Hurt |
| Use 2 layers per block rather than the selected 3-layer blocks (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 15.98 | Hurt |
| Replace distance-aware Gaussian weighting with self-attention aggregation (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 13.90 | Hurt |
| Replace distance-aware Gaussian weighting with concatenation (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 15.68 | Hurt |
| Replace distance-aware Gaussian weighting with sum aggregation (ablation) | SGC-Net · SWIG-HOI Full | mAP: 17.20 -> 15.26 | Hurt |
| Make DGW weights fixed instead of trainable (ablation) | SGC-Net · SWIG-HOI Full | mAP: 16.71 -> 17.20 | Hurt |
Experimental setup
Datasets: HICO-DET SWIG-HOI
Benchmarks: HICO-DET SWIG-HOI
Metrics: mAP
Baselines: CLIP-based OV-HOI detectors GEN-VLKT OpenCat HOICLIP CDN-based detectors
Builds on: CLIP HOICLIP CLIP-based open-vocabulary HOI detection
Closest papers by meaning
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.91)
- Toward Open-Set Human Object Interaction Detection (AAAI 2024 · similarity 0.88)
- Open-World Human-Object Interaction Detection via Multi-modal Prompts (CVPR 2024 · similarity 0.88)
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.88)
- Discovering Syntactic Interaction Clues for Human-Object Interaction Detection (CVPR 2024 · similarity 0.87)
- Visual Diversity and Region-aware Prompt Learning for Zero-shot HOI Detection (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.