Research map / 3D vision / Human pose estimation
MRC-Net: 6-DoF Pose Estimation with MultiScale Residual Correlation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MRC-Net, a single-shot sequential Siamese network that classifies coarse 6-DoF pose with soft symmetry-aware labels, renders the object, and regresses residual pose using multi-scale residual correlation features, achieving state-of-the-art RGB-based BOP results.
Problem
Single-shot 6-DoF object pose estimation from a single RGB image given a known CAD model, where discrete pose classification loses accuracy through quantization and parallel regression receives no guidance from classification.
Approach
Sequentially chain a pose classifier and a residual regressor as Siamese networks with shared weights, bridged by a multi-scale residual correlation (MRC) layer that correlates real-image and rendered-image features at three scales, plus soft probabilistic (symmetry-aware) class labels for classification.
Key results
- Trained on synthetic PBR only, MRC-Net reaches 77.1 AR on T-LESS, 39.3 on ITODD, 68.1 on YCB-V and 68.5 on LM-O, averaging 63.3 and outperforming listed RGB methods.
- Soft labels improve average AR by 2.3% over hard labels on T-LESS (77.1 vs 74.8).
- Removing the parallel-to-sequential design costs 11.3% AR; MRC multi-scale correlation adds 0.6% over single-scale correlation on T-LESS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use soft rather than hard pose-class labels (ablation) | MRC-Net · T-LESS | AR: 74.8 -> 77.1; increase of 2.3% | Helped |
| Add a parallel regression head to the classification-only model (ablation) | MRC-Net · T-LESS | AR: 65.3 -> 64.9 | No clear effect |
| Use parallel rather than sequential classification and regression (ablation) | MRC-Net · T-LESS | AR: 77.1 -> 64.9; 11.3% in average AR | Hurt |
| Condition residual regression on classification using sequential feature concatenation (ablation) | MRC-Net · T-LESS | AR: 64.9 -> 76.2 | Helped |
| Replace feature concatenation with single-scale feature correlation (ablation) | MRC-Net · T-LESS | AR: 76.2 -> 76.5; 0.3% additional improvement | Helped |
| Extend single-scale correlation to multi-scale correlation (ablation) | MRC-Net · T-LESS | AR: 76.5 -> 77.1; 0.6% | Helped |
| Add perspective correction (ablation) | MRC-Net · T-LESS | AR: 76.6 -> 77.1; 0.5% | Helped |
| Add test-time augmentation (ablation) | MRC-Net · T-LESS | AR: 76.7 -> 77.1; 0.4% | Helped |
Surprising findings
- A parallel multitask regression head added to the classifier gives little or no improvement over classification alone.
Limitations
- Degrades when CAD models are inaccurate, since it relies on render-and-compare.
- Lags top methods on fine-tuned YCB-V by up to 0.9% AR, attributed to rich textures favoring correspondence methods and noisy annotations.
- Focuses only on RGB input; depth is not used.
Experimental setup
Datasets: T-LESS LM-O YCB-V ITODD PBR synthetic images (BOP)
Benchmarks: T-LESS ITODD YCB-V LM-O
Metrics: Average Recall (AR) ARVSD ARMSSD ARMSPD ADD(-S) AUC of ADD-S AUC of ADD(-S)
Baselines: EPOS CDPNv2 DPODv2 PVNet CosyPose SurfEmb SC6D SCFlow CIR PFA ZebraPose CheckerPose
Builds on: render-and-compare SC6D SITE focal loss CosyPose
Closest papers by meaning
- DSC-PoseNet: Learning 6DoF Object Pose Estimation via Dual-Scale Consistency (CVPR 2021 · similarity 0.90)
- One2Any: One-Reference 6D Pose Estimation for Any Object (CVPR 2025 · similarity 0.87)
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.86)
- GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.86)
- FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation (CVPR 2024 · similarity 0.86)
- SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.