Research map / 3D vision / Face and shape

PICO: Reconstructing 3D People In Contact with Objects

CVPR 2025 · 3D vision › Face and shape · approach: Full-body avatar reconstruction · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds PICO-db, a dataset pairing in-the-wild images with dense body and object contact correspondences (via retrieved Objaverse meshes and two-click contact transfer from DAMON), and PICO-fit, an optimization-based render-and-compare method that fits 3D human and object meshes using contact constraints.

Problem

Recovering 3D human-object interaction from single in-the-wild color images is hard due to depth ambiguity, occlusion, and object shape variety; existing methods work only in controlled settings or for limited object classes, and no dataset pairs natural images with contacts on both body and object.

Approach

Repurpose DAMON's body-only contact patches by transferring them onto retrieved 3D object meshes with a PCA-generated contact axis requiring just two clicks, yielding bijective body-object contact correspondences; then use these contacts in a three-stage render-and-compare fitting (object-to-body registration, object mask alignment, limb pose refinement) that generalizes to novel object classes vi

Key results

• On InterCap, PICO-fit (no GT contact) achieves PA-CD of 7.43 cm (human) and 10.33 cm (human+object) versus HDM 17.34 and 13.6; with GT contact PICO-fit* reaches 6.66 / 8.36.
• In an AMT perceptual study on 75 DAMON images, PICO-fit* was preferred over baselines 74.4% of the time on average.
• PICO-db contains 4123 annotated images spanning 44 object categories and 627 object instances.

Measured effects reported in the paper

Surprising findings

• Adding an extra joint refinement stage did not help according to the authors' empirical finding, so they keep object and body refinement separate.

Limitations

• Contact initialization relies on noisy DECO predictions and GPT-4V, and nearest-neighbor retrieval from a finite dataset limits generalization.
• Object shape is retrieved from a database rather than reconstructed, so unseen shapes are only approximated.
• Evaluation on in-the-wild images is via a perceptual study, not ground-truth 3D metrics.

Experimental setup

Datasets: DAMON Objaverse-LVIS InterCap BEHAVE

Benchmarks: InterCap DAMON

Metrics: Procrustes-aligned Chamfer distance preference rate

Baselines: PHOSA CONTHO HDM

Builds on: DAMON ContactEdit OSX OpenShape DECO

Closest papers by meaning

• Reconstructing In-the-Wild Open-Vocabulary Human-Object Interactions (CVPR 2025 · similarity 0.90)
• InteractVLM: 3D Interaction Reasoning from 2D Foundational Models (CVPR 2025 · similarity 0.89)
• InteractAnything: Zero-shot Human Object Interaction Synthesis via LLM Feedback and Object Affordance Parsing (CVPR 2025 · similarity 0.88)
• EasyHOI: Unleashing the Power of Large Models for Reconstructing Hand-Object Interactions in the Wild (CVPR 2025 · similarity 0.88)
• InterAct: Advancing Large-Scale Versatile 3D Human-Object Interaction Generation (CVPR 2025 · similarity 0.87)
• Joint Reconstruction of 3D Human and Object via Contact-Based Refinement Transformer (CVPR 2024 · similarity 0.87)