Research map / 3D vision / Face and shape
KP-RED: Exploiting Semantic Keypoints for Joint 3D Shape Retrieval and Deformation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KP-RED, a keypoint-driven framework that jointly retrieves a CAD model from a database and deforms it via neural cage deformation to match full or partial object scans.
Problem
Retrieval and deformation (R&D) of CAD models from noisy, partial object scans; existing global-feature retrieval and dense point matching are sensitive to occlusion and outliers.
Approach
Uses unsupervised category-consistent sparse keypoints as the intermediate representation: local-global keypoint-region tokens for retrieval, density-weighted keypoint aggregation for partial scans, and self-attention over keypoint regions to predict cage influence vectors for deformation.
Key results
- On full PartNet shapes, KP-RED average CD 0.089 vs. 0.340 for ShapeFlow and 0.551 for U-RED (lowest among compared methods).
- On Scan2CAD partial scans, average UCD 0.060 vs. 0.210 for Uy et al., 0.207 for U-RED, and 0.265 for ShapeFlow.
- Inference runs at about 30 ms per instance, versus 0.7 s for Uy et al. and about 45 s for ShapeFlow.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use KP-RED instead of ShapeFlow for joint retrieval and deformation | PartNet full shapes · PartNet | Chamfer Distance: 0.238 -> 0.091 | Helped |
| use KP-RED instead of Uy et al. for retrieval and deformation | Scan2CAD object scans · Scan2CAD | Unilateral Chamfer Distance: 0.158 -> 0.059 | Helped |
| add geometric self-attention to the encoder (ablation) | KP-RED · PartNet full shapes | Average Chamfer Distance: 0.161 -> 0.110 | Helped |
| add deformation-aware retrieval (ablation) | KP-RED with geometric self-attention and no local-global feature embedding · PartNet full shapes | Average Chamfer Distance: 0.110 -> 0.098 | Helped |
| add local-global feature embedding (ablation) | KP-RED with geometric self-attention and deformation-aware retrieval · PartNet full shapes | Average Chamfer Distance: 0.098 -> 0.089 | Helped |
| add confidence-based dynamic feature extraction (ablation) | KP-RED · Partial PartNet shapes | Unilateral Chamfer Distance: 11% for up to 75% occlusion; 8% for less than 50% occlusion | Helped |
Surprising findings
- Average UCD on partial PartNet rises only 30.6% from 25% to 75% occlusion for KP-RED, versus about 60-70% for baselines.
Limitations
- Evaluated only on three categories (chair, table, cabinet).
- Partial inputs simulated by random slicing on PartNet rather than real scans except Scan2CAD evaluation.
- Depends on a pose estimator for canonicalization of the target.
Experimental setup
Datasets: PartNet Scan2CAD ShapeNet
Benchmarks: PartNet Scan2CAD augmented partial PartNet
Metrics: Chamfer Distance Unilateral Chamfer Distance
Baselines: Uy et al. U-RED ShapeFlow
Builds on: neural cage deformation (Wang et al. 2019 / Yifan et al. 2020) keypoint-driven cage deformation (KeypointDeformer, Jakab et al.) Uy et al. joint retrieval and deformation PointNet U-RED
Closest papers by meaning
- KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control (CVPR 2021 · similarity 0.90)
- Joint Learning of 3D Shape Retrieval and Deformation (CVPR 2021 · similarity 0.89)
- ShapeMatcher: Self-Supervised Joint Shape Canonicalization Segmentation Retrieval and Deformation (CVPR 2024 · similarity 0.87)
- Unsupervised 3D Structure Inference from Category-Specific Image Collections (CVPR 2024 · similarity 0.86)
- Key-Grid: Unsupervised 3D Keypoints Detection using Grid Heatmap Features (NeurIPS 2024 · similarity 0.86)
- Back to 3D: Few-Shot 3D Keypoint Detection with Back-Projected 2D Features (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.