Research map / Detection and segmentation / Open-set and few-shot detection
Dual-Prior Augmented Decoding Network for Long Tail Distribution in HOI Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a dual-prior augmented decoding network that decomposes HOI detection into object-level and verb-level subtasks, injecting external classifier priors (MLDecoder objects, BLIP-VQA verbs) into cascaded decoders to better recognize tail HOI categories.
Problem
Long-tailed distribution in HOI detection makes tail categories hard to recognize, and existing external-knowledge methods (e.g., VLM distillation) only use knowledge at the HOI combination level, giving limited tail gains.
Approach
Decompose HOI into human-object pair detection and interaction recognition, acquire object and verb prior candidates from external classifiers (MLDecoder and BLIP VQA), filter them with cross-attention, embed them into visual features, and decode with conditional DETR-style cascaded decoders, plus BLIP-based VLKT at the combination level.
Key results
- On HICO-DET default setting, the method reaches 35.91 full mAP and 35.82 rare mAP, versus 33.75 full and 29.25 rare for the Gen-VLKT baseline.
- Under RF-UC zero-shot on HICO-DET, the method achieves 31.83 unseen mAP, a +10.47 gain over Gen-VLKT.
- On V-COCO the method reports AP role 62.68, surpassing Gen-VLKT but with a smaller margin than on HICO-DET.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dual-prior augmented decoding to Gen-VLKT | Gen-VLKT on HICO-DET Rare · HICO-DET Rare | mAP: 29.25 -> 35.82 | Helped |
| add dual-prior augmented decoding to Gen-VLKT | Gen-VLKT on HICO-DET Full · HICO-DET Full | mAP: 33.75 -> 35.91 | Helped |
| add dual-prior augmented decoding to Gen-VLKT | Gen-VLKT in RF-UC · HICO-DET RF-UC unseen | mAP: 21.36 -> 31.83 | Helped |
| add dual-prior augmented decoding to Gen-VLKT | Gen-VLKT in UV · HICO-DET UV unseen | mAP: 20.96 -> 27.45 | Helped |
| replace CLIP-based VLKT with BLIP-based VLKT (ablation) | Gen-VLKT on HICO-DET · HICO-DET Rare | mAP: 29.25 -> 30.49 | Helped |
| replace CLIP-based VLKT with BLIP-based VLKT (ablation) | Gen-VLKT on HICO-DET · HICO-DET Non-Rare | mAP: 35.01 -> 34.86 | Hurt |
| use classifier-weight prior encoding instead of random encoding (ablation) | DP-ADN · HICO-DET Full | mAP: 34.77 -> 35.91 | Helped |
| use classifier-weight prior encoding instead of BLIP text embeddings (ablation) | DP-ADN · HICO-DET Full | mAP: 35.35 -> 35.91 | Helped |
| add object-prior acquisition (ablation) | DP-ADN baseline (BLIP-based VLKT) · HICO-DET Rare | mAP: 30.49 -> 31.47 | Helped |
| add object-prior embedding (ablation) | DP-ADN with object-prior acquisition · HICO-DET Full | mAP: 33.93 -> 34.34 | Helped |
| add verb-prior embedding (ablation) | DP-ADN with object-prior acquisition and embedding · HICO-DET Rare | mAP: 31.67 -> 33.73 | Helped |
| add verb-prior acquisition (ablation) | DP-ADN with object and verb prior embedding · HICO-DET Rare | mAP: 33.73 -> 35.10 | Helped |
Surprising findings
- Replacing CLIP with BLIP in the VLKT module alone raised rare mAP from 29.25 to 30.49 while lowering non-rare mAP (35.01 to 34.86).
- The conditional decoder was not used on V-COCO because many examples lack interacting objects, making reference-point-based detection difficult.
Limitations
- Improvement on V-COCO is less significant than on HICO-DET due to its smaller scale and simpler interaction set.
- NF-UC zero-shot result (25.67 full) is below HOICLIP (27.75).
Experimental setup
Datasets: HICO-DET V-COCO
Benchmarks: HICO-DET V-COCO
Metrics: mAP AP role
Baselines: Gen-VLKT HOICLIP CDN ViPLO PartMap CATN QPIC VCL ATL FCL
Builds on: Gen-VLKT CDN Conditional DETR HOICLIP CATN
Closest papers by meaning
- Detecting Any Human-Object Interaction Relationship: Universal HOI Detector with Spatial Prompt Learning on Foundation Models (NeurIPS 2023 · similarity 0.90)
- Human-Object Interaction Detection via Disentangled Transformer (CVPR 2022 · similarity 0.90)
- HORP: Human-Object Relation Priors Guided HOI Detection (CVPR 2025 · similarity 0.88)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.88)
- Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.88)
- HOIMamba: Efficient Mamba-based Disentangled Progressive Learning for HOI Detection (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.