Research map / Vision-language and multimodal / Multimodal LLMs
Unleashing Multimodal Large Language Models for Training-free HOI Detection in the Wild
Ting Lei, Jialin Liu, Zhu Xu, Yuxin Peng, Yang Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add context-aware multi-round reasoning (CMR) (ablation) | AgentHOI-Base · HICO-DET Full | mAP: 19.44 -> 23.20 | Improved |
| add multifaceted interaction localization (MIL) on top of CMR (ablation) | AgentHOI-Base + CMR · HICO-DET Full | mAP: 23.20 -> 29.61 | Improved |
| add HOI re-mining to initial identification (object recognition) (ablation) | AgentHOI · HICO-DET images | F1: 78.01 -> 83.15 | Improved |
| add action reassignment after HR (HOI recognition) (ablation) | AgentHOI · HICO-DET images | F1: 47.21 -> 64.91 | Improved |
| add action reassignment to IHI+HR (action recognition) (ablation) | AgentHOI · HICO-DET images | F1: 51.34 -> 65.72 | Improved |
| use more re-mining iterations (three instead of one) (ablation) | AgentHOI · HICO-DET HOI recognition | F1: 64.91 -> 64.40 | No clear effect |
| combine description and relation prompts instead of description-only (ablation) | AgentHOI · HICO-DET Full | mAP: 27.70 -> 29.61 | Improved |
| use description-only instead of relation-only localization (ablation) | AgentHOI · HICO-DET Full | mAP: 24.30 -> 27.70 | Improved |
| use GPT-4o instead of Qwen2.5-VL-72B as MLLM (ablation) | AgentHOI · HICO-DET Full | mAP: 26.12 -> 29.61 | Improved |
| set lambda to 0.3 instead of 0.1 (ablation) | AgentHOI · HICO-DET Full | mAP: 29.42 -> 29.61 | Improved |
| increase lambda from 0.3 to 0.5 (ablation) | AgentHOI · HICO-DET Full | mAP: 29.61 -> 29.44 | Hurt |
| add CMR and MIL to AgentHOI-Base on SWIG-HOI (ablation) | AgentHOI-Base · SWIG-HOI Unseen | mAP: 8.74 -> 11.61 | Improved |
Closest papers by meaning
- Open-World Human-Object Interaction Detection via Multi-modal Prompts (CVPR 2024 · similarity 0.93)
- InstructHOI: Context-Aware Instruction for Multi-Modal Reasoning in Human-Object Interaction Detection (NeurIPS 2025 · similarity 0.93)
- Learning to Diversify and Focus: A Reinforcement Framework for Open-Vocabulary HOI Detection (CVPR 2026 · similarity 0.91)
- Exploring the Potential of Large Foundation Models for Open-Vocabulary HOI Detection (CVPR 2024 · similarity 0.90)
- Learning Human-Object Interaction as Groups (NeurIPS 2025 · similarity 0.89)
- Human-Object Interaction Detection Collaborated with Large Relation-driven Diffusion Models (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
