Research map / 3D vision / Human pose estimation
CRISP: Object Pose and Shape Estimation with Test-Time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRISP, a category-agnostic RGB-D object pose and shape estimator with an optimization-based active-shape corrector and correct-and-certify self-training (CRISP-ST) for test-time domain adaptation.
Problem
Estimating object 6D pose and shape from RGB-D without category priors or object onboarding, while remaining robust to domain gaps at test time.
Approach
Encoder-decoder shape model (DINOv2 ViT + FiLM-conditioned SDF decoder) plus DPT-predicted pose-normalized coordinates; an active shape model reduces shape correction to a constrained linear least squares problem, and an observable correctness certificate filters corrected pseudo-labels for self-training without synthetic data.
Key results
- On YCBV, CRISP-Real achieves mean eshape 0.026 vs Shap-E 0.099 (about 73% lower).
- On SPE3R, CRISP-ST improves mean eL1 shape to 0.141 from SQRECON's 0.145 and mean eL1 pose to 0.224 from 0.304.
- On REAL275, CRISP reaches 83.5 mAP at IoU50 and 70.5 at IoU75, best among compared methods, and achieves 67% shape error reduction over CASS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| self-train CRISP-Syn with the LSQ corrector | CRISP-Syn · YCBV | eshape mean: 0.045 -> 0.037 | Helped |
| self-train CRISP-Syn with the LSQ corrector | CRISP-Syn · YCBV | ADD-S AUC (2 cm): 0.30 -> 0.44 | Helped |
| self-train CRISP on the test set | CRISP · SPE3R | eL1 shape mean: 0.163 -> 0.141 | Helped |
| self-train CRISP on the test set | CRISP · SPE3R | eL1 pose mean: 0.292 -> 0.224 | Helped |
| add projection onto the latent-shape simplex during BCD (ablation) | CRISP-Syn-ST · YCBV test-time adaptation | eshape AUC (5 cm): 0.02 -> 0.39 | Helped |
| use a multi-view rather than single-view corrector (ablation) | CRISP-Syn-ST · YCBV test-time adaptation | eshape AUC (5 cm): 0.29 -> 0.39 | Helped |
| add a corrector before certification and self-training (ablation) | CRISP-Syn-ST · YCBV test-time adaptation | ADD-S AUC (2 cm): 0.30 -> 0.42 | Helped |
| use certified pseudo-label self-training instead of direct self-supervision (ablation) | CRISP-Syn-ST · YCBV test-time adaptation | ADD-S AUC (2 cm): 0.29 -> 0.42 | Helped |
| use certified pseudo-label self-training instead of direct self-supervision (ablation) | CRISP-Syn-ST · YCBV test-time adaptation | eshape AUC (5 cm): 0.38 -> 0.39 | No clear effect |
| train CRISP on REAL275 | CRISP · REAL275 | pose mAP (IoU50): 81.1 -> 83.5 | Helped |
| train CRISP on REAL275 | CRISP · REAL275 | pose mAP (10° 5 cm): 70.7 -> 62.8 | Hurt |
Surprising findings
- Self-training without the corrector passes far fewer pseudo-labels (6.1% of outputs pass the certificate vs 16.6% with BCD corrector).
- On REAL275, CRISP's rotation error is higher than category-level baselines such as RePoNet and SGPA, with 5deg 5cm accuracy of 22.5.
Limitations
- Assumes ground-truth object data association at test time.
- Latent shape code basis used in the corrector scales linearly with the number of training objects.
- Rotation error is higher than category-level baselines on REAL275.
Experimental setup
Datasets: YCBV SPE3R NOCS CAMERA NOCS REAL275
Benchmarks: YCBV SPE3R NOCS REAL275
Metrics: Chamfer distance (eshape) ADD-S ADD-S AUC mAP at IoU50/IoU75 5deg 5cm 10deg 5cm
Baselines: Shap-E CosyPose BundleSDF GDRNet++ GSPose FoundationPose SQRECON SPD SGPA CASS RePoNet DualPoseNet
Builds on: NOCS DPT FiLM correct-and-certify self-training DINOv2
Closest papers by meaning
- GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.85)
- NOPE: Novel Object Pose Estimation from a Single Image (CVPR 2024 · similarity 0.85)
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.85)
- MFOS: Model-Free & One-Shot Object Pose Estimation (AAAI 2024 · similarity 0.85)
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.84)
- One2Any: One-Reference 6D Pose Estimation for Any Object (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.