Research map / 3D vision / Human pose estimation
Leveraging Global Stereo Consistency for Category-Level Shape and 6D Pose Estimation from Stereo Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a stereo-based category-level shape and 6D pose/scale estimator that fits a shared intra-category occupancy field to both views via iterative global shape fitting (GSF-Net), and introduces the S-RGBD dataset for comparing stereo and RGBD methods across depth-friendly and depth-challenging materials.
Problem
Category-level shape and 6D pose estimation from stereo images, where missing explicit depth causes pose, scale and shape parameters to become entangled and optimization to fall into local optima; also whether stereo or RGBD methods better handle diverse materials with poor depth.
Approach
Use a shared intra-category occupancy field across left and right views plus a divide-and-conquer global shape fitting network that iteratively renders normalized depth maps and shares rotation, y-translation and lambda (s/z) across views, with independent x-offsets and stereo-disparity depth to resolve scale.
Key results
- On S-RGBD, achieves 3D_avg of 82.9 on bowl and 89.7 on box, outperforming stereo baseline StereoPose on all 10 categories (e.g., mug 80.2 vs 64.8).
- On TOD bottle, 3D50 of 84.2 and 3D25 of 64.1 for mug, surpassing CODERS (73.4 and 31.5 respectively for mug 3D25/3D50 per table columns).
- Ablation on S-RGBD: full model reaches 95.6 3D25, 83.7 3D50, 60.5 3D75; removing lambda constraint drops 3D75 to 44.2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add global stereo consistency with global shape fitting | Ours · TOD Bottle | 3D50: 73.4 -> 84.2 | Helped |
| Add global stereo consistency with global shape fitting | Ours · TOD Mug | 3D50: 92.8 -> 95.9 | Helped |
| Add the shared intra-category shape representation and global stereo fitting | Ours · S-RGBD Knife | 3Davg: 58.1 -> 67.9 | Helped |
| Add the shared intra-category shape representation and global stereo fitting | Ours · S-RGBD Spoon | 3Davg: 54.3 -> 63.5 | Helped |
| Use the monocular version instead of stereo (ablation) | Ours · S-RGBD | 3D50: 83.7 -> 30.5 | Hurt |
| Remove global fitting (ablation) | Ours · S-RGBD | 3D50: 83.7 -> 32.4 | Hurt |
| Add the shape constraint (ablation) | Ours · S-RGBD | 3D50: 78.1 -> 83.7 | Helped |
| Add the rotation constraint (ablation) | Ours · S-RGBD | 3D50: 76.5 -> 83.7 | Helped |
| Add the y-translation constraint (ablation) | Ours · S-RGBD | 3D50: 77.5 -> 83.7 | Helped |
| Add the λ constraint (ablation) | Ours · S-RGBD | 3D50: 71.1 -> 83.7 | Helped |
| Use semantic features only, removing geometric features (ablation) | Ours · S-RGBD | 3D50: 83.7 -> 74.6 | Hurt |
| Use geometric features only, removing semantic features (ablation) | Ours · S-RGBD | 3D50: 83.7 -> 77.7 | Hurt |
Surprising findings
- Tuned RGBD methods (Second) perform strongly on depth-friendly objects but drop sharply on depth-challenging ones (e.g., mug 78.8 vs 26.3 3D_avg), while the stereo method is more material-robust.
Limitations
- Failure cases with transparent objects reflecting surroundings, perspective-induced height errors, and occlusion ambiguity.
- Very poor-quality masks cause failure.
- Scale ambiguity is not fully resolved by lambda alone, requiring independent x-offsets.
Experimental setup
Datasets: TOD S-RGBD
Benchmarks: TOD S-RGBD
Metrics: 3D IoU at 25/50/75 3D_avg rotation error 5°/10° translation error 2/5/10 cm grasp success rate
Baselines: SPD SGPA Second KeyPose StereoPose CODERS
Builds on: NOCS ZeroShape CODERS Megapose DINOv2
Closest papers by meaning
- SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (NeurIPS 2025 · similarity 0.88)
- SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.87)
- GCE-Pose: Global Context Enhancement for Category-level Object Pose Estimation (CVPR 2025 · similarity 0.86)
- MVD-Fusion: Single-view 3D via Depth-consistent Multi-view Generation (CVPR 2024 · similarity 0.86)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.86)
- Stereo Anywhere: Robust Zero-Shot Deep Stereo Matching Even Where Either Stereo or Mono Fail (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.