Research map / 3D vision / Human pose estimation
MatchU: Matching Unseen Objects for 6D Pose Estimation from RGB-D Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
MatchU fuses RGB and depth/CAD geometry with rotation-invariant 3D descriptors and a bridged coarse-level matching loss to estimate 6D poses of unseen objects by descriptor matching, trained on GSO and evaluated on five BOP core datasets.
Problem
6D pose estimation for previously unseen objects from single RGB-D images, without per-object retraining or costly rendering-based refinement, where geometry-only correspondences are ambiguous due to symmetry.
Approach
Learn rotation-invariant (PPF-based) 3D descriptors and fuse 2D texture into them via a Latent Fusion Attention Module, supervised by a Bridged Coarse-level Matching loss that uses RGB latent features as a bridge between CAD and depth point cloud latent spaces.
Key results
- Ours (Accurate) with refinement reaches 64.7 mean AR on BOP-5 unseen-localization setting, versus 63.9 for MegaPose (BOP) and 56.4 for ZeroPose (BOP).
- Ours (Fast) with one hypothesis and no refinement reaches 50.5 mean AR in 0.07 s per inference.
- On LM-O with ADD-0.1d, Ours (20 hypotheses) reaches 70.8 under unseen localization and 75.7 under seen localization, close to DFTr's 77.7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add BCM Loss (ablation) | MatchU · BOP-5 | mean AR: 48.1 -> 50.5 | Helped |
| Add RGB input (ablation) | MatchU · BOP-5 | mean AR: 39.6 -> 50.5 | Helped |
| Replace RoITr with MatchU's fusion pipeline (ablation) | RGB-initialized RoITr · BOP-5 | mean AR: 42.6 -> 50.5 | Helped |
| Increase the number of pose hypotheses from 1 to 20 (ablation) | MatchU · LM-O | AR: 52.6 -> 56.2 | Helped |
| Increase the number of pose hypotheses beyond 50 (ablation) | MatchU · LM-O | AR: 58.3 -> 58.3 | No clear effect |
Surprising findings
- Trained without test objects, the method outperforms most baselines that have seen them, e.g., PVN3D and FFB6D on LM-O.
Limitations
- Relies on external object localization (segmentation/detection), and errors in it propagate to pose estimation.
- Not end-to-end trainable in current pipeline.
- Performance gap to object-specific methods remains (per authors' framing).
Experimental setup
Datasets: Google-Scanned-Objects (GSO) rendered by MegaPose LM-O T-LESS TUD-L IC-BIN YCB-V
Benchmarks: BOP-5 core (LM-O, T-LESS, TUD-L, IC-BIN, YCB-V) LM-O
Metrics: Average Recall (AR) ADD-0.1d time (s)
Baselines: ZeroPose OSOP MegaPose OVE6D GCPose PVN3D FFB6D DFTr DrostPPF PPF + Zephyr
Builds on: RoITr LoFTR GeoTransformer Circle Loss Point Pair Features
Closest papers by meaning
- SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (NeurIPS 2025 · similarity 0.89)
- RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects (CVPR 2025 · similarity 0.88)
- Pos3R: 6D Pose Estimation for Unseen Objects Made Easy (CVPR 2025 · similarity 0.87)
- One2Any: One-Reference 6D Pose Estimation for Any Object (CVPR 2025 · similarity 0.87)
- HiPose: Hierarchical Binary Surface Encoding and Correspondence Pruning for RGB-D 6DoF Object Pose Estimation (CVPR 2024 · similarity 0.87)
- Any6D: Model-free 6D Pose Estimation of Novel Objects (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.