Research map / Detection and segmentation / Open-set and few-shot detection

HOComp: Interaction-Aware Human-Object Composition

NeurIPS 2025 · Detection and segmentation › Open-set and few-shot detection · approach: Hand-object and articulated interaction · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HOComp, a FLUX.1-based LoRA-tuned framework that composes a foreground object onto a human-centric background with MLLM-guided interaction regions, pose loss, and appearance-preservation losses, plus a new IHOC dataset and HOIBench evaluation.

Problem

Compositing a foreground object onto a human background so that the human-object interaction is natural while foreground object identity and background person appearance remain consistent; existing image-guided composition methods often produce implausible gestures and change identities.

Approach

Use an MLLM (GPT-4o) in a multi-stage query to pick the interaction type, object box, and interaction region as coarse constraints, add a region-restricted human pose keypoint loss as fine constraint, and preserve appearance via shape-aware residual attention modulation, a multi-view CLIP appearance loss, and a background consistency loss.

Key results

• Achieves FID 9.27, CLIP-Score 30.29, HOI-Score 87.39, DINO-Score 78.21, SSIM(BG) 96.57 on HOIBench, outperforming GPT-4o (HOI 75.22, DINO 65.23).
• User study over 75 participants ranks the method first in image quality (1.37), interaction harmonization (1.14), and appearance preservation (1.11).
• Removing the background consistency loss drops SSIM(BG) from 96.57 to 58.49.

Measured effects reported in the paper

Surprising findings

• Naively increasing the pose loss weight to 1.5 improves HOI score but degrades FID, DINO and SSIM(BG), indicating a trade-off.
• GPT-4o achieves a strong HOI score but has low background SSIM (47.22), showing it regenerates the background.

Limitations

• MLLM identifies the correct interaction region in only 91.33% of HOIBench samples, and errors degrade the output.
• Evaluation is on a self-constructed benchmark of 600 instances from 30 human images.
• Interaction region prediction does not yet use human pose priors.

Experimental setup

Datasets: HICO-DET IHOC

Benchmarks: HOIBench

Metrics: FID CLIP-Score HOI-Score DINO-Score SSIM(BG) user study rank (IQ, IH, AP)

Baselines: AnyDoor Paint by Example FreeCompose FreeCustom PrimeComposer OmniGen GenArtist UniCombine GPT-4o DreamFuse InsertAnything MimicBrush

Builds on: FLUX.1 [dev] LoRA DWPose Zero123+ HICO-DET

Closest papers by meaning

• Putting People in Their Place: Affordance-Aware Human Insertion Into Scenes (CVPR 2023 · similarity 0.87)
• InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.85)
• Person in Place: Generating Associative Skeleton-Guidance Maps for Human-Object Interaction Image Editing (CVPR 2024 · similarity 0.85)
• Exploring Pose-Aware Human-Object Interaction via Hybrid Learning (CVPR 2024 · similarity 0.85)
• IMAGPose: A Unified Conditional Framework for Pose-Guided Person Generation (NeurIPS 2024 · similarity 0.84)
• AvatarGO: Zero-shot 4D Human-Object Interaction Generation and Animation (ICLR 2025 · similarity 0.84)