Research map / Detection and segmentation / Open-set and few-shot detection
Precision-Enhanced Human-Object Contact Detection via Depth-Aware Perspective Interaction and Object Texture Restoration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PIHOT, a human-object contact detector that combines object inpainting (LaMa) with depth-map differencing (ZoeDepth) and attention-based instance interaction modules to segment contact areas under occlusion.
Problem
Human-object contact (HOT) detection fails when objects are occluded by humans or other objects, causing false contact detections and inaccurate contact boundaries in 2D images.
Approach
Restore occluded object textures via inpainting of dilated human masks and use the depth difference between original and inpainted images to determine real spatial contact, fusing these with attention-based instance perspective and depth-space interaction modules.
Key results
- On HOT-Annotated, PIHOT reaches SC-Acc. 45.3, C-Acc. 80.7, mIoU 0.236, wIoU 0.286, versus DHOT Full at 40.7, 70.7, 0.215, 0.260.
- On HOT-Generated, PIHOT achieves 34.9, 76.3, 0.169, 0.212, outperforming DHOT by 15%, 41%, 22%, 27% on the four metrics.
- On the Full Set, PIHOT reaches 42.4 SC-Acc., 77.5 C-Acc., 0.221 mIoU, 0.273 wIoU; the paper reports improvements of 16%, 17%, 6%, 9% over the second-best method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace DHOT Full with PIHOT | HOT detection · HOT-Annotated | C-Acc.: 70.7 -> 80.7 | Helped |
| Replace DHOT Full with PIHOT | HOT detection · HOT-Generated | C-Acc.: 54.3 -> 76.3 | Helped |
| Add OI (object inpainting) to the baseline (ablation) | PIHOT · HOT-Annotated | SC-Acc.: 40.5 -> 42.4 (+7%) | Helped |
| Add OI (object inpainting) to the baseline (ablation) | PIHOT · HOT-Generated | mIoU: 0.135 -> 0.128 (-7%) | Hurt |
| Add OI (object inpainting) to the baseline (ablation) | PIHOT · Full Set | wIoU: 0.194 -> 0.263 (+36%) | Helped |
| Add IPI to OI (ablation) | PIHOT · HOT-Annotated | SC-Acc.: 42.4 -> 44.2 (+4%) | Helped |
| Add IPI to OI (ablation) | PIHOT · HOT-Annotated | wIoU: 0.269 -> 0.265 (-1%) | Hurt |
| Add IPI to OI (ablation) | PIHOT · Full Set | SC-Acc.: 38.4 -> 40.7 (+6%) | Helped |
| Add SPO to OI + IPI (ablation) | PIHOT · HOT-Annotated | mIoU: 0.221 -> 0.231 (+5%) | Helped |
| Add SPO to OI + IPI (ablation) | PIHOT · HOT-Generated | mIoU: 0.134 -> 0.169 (+24%) | Helped |
| Add SPO to OI + IPI (ablation) | PIHOT · Full Set | wIoU: 0.261 -> 0.270 (+3%) | Helped |
| Add IDSI to OI + IPI + SPO (ablation) | PIHOT · HOT-Annotated | C-Acc.: 79.7 -> 80.7 (+1%) | Helped |
Surprising findings
- Replacing the full pipeline with MAE+MDENet yields only slightly lower results on some metrics; mIoU is higher (0.279) than the selected LaMa+ZoeDepth (0.236) for that variant.
- Adding the IPI module reduced wIoU by about 1% on HOT-Annotated, showing the module does not improve all metrics.
Limitations
- May be limited to extreme occlusion scenarios or dynamic scenes.
- Relies on external inpainting and depth models whose errors propagate into the pipeline.
- Human masks at inference can come from external segmentation models rather than annotations.
Experimental setup
Datasets: HOT-Annotated HOT-Generated V-COCO HAKE Watch-n-Patch PROX SMPL-X
Benchmarks: HOT-Annotated HOT-Generated Full Set
Metrics: SC-Acc. C-Acc. mIoU wIoU
Baselines: ResNet+UperNet ResNet+PPM DHOT
Builds on: DHOT LaMa ZoeDepth
Closest papers by meaning
- PICO: Reconstructing 3D People In Contact with Objects (CVPR 2025 · similarity 0.84)
- Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer (CVPR 2024 · similarity 0.84)
- InteractVLM: 3D Interaction Reasoning from 2D Foundational Models (CVPR 2025 · similarity 0.84)
- Bilateral Adaptation for Human-Object Interaction Detection with Occlusion-Robustness (CVPR 2024 · similarity 0.83)
- ContactField: Implicit Field Representation for Multi-Person Interaction Geometry (NeurIPS 2024 · similarity 0.82)
- InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.