Research map / 3D vision / Human pose estimation
MFOS: Model-Free & One-Shot Object Pose Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MFOS, a fully transformer-based, model-free one-shot object pose estimator that predicts dense 2D-3D maps to a rendered cuboid proxy shape from a query image and annotated reference views, initialized from CroCo v2 pretraining.
Problem
Estimating 6D pose of unseen objects from RGB images at test time without 3D models, depth, or video, given only a few posed reference images and an approximate object size.
Approach
A ViT encoder-decoder that injects reference poses by rendering a cuboid proxy shape's coordinates as an image, cross-attends query to all reference views, and regresses proxy-shape 3D coordinates with confidence, followed by PnP; geometric pretraining (CroCo) is key to generalization.
Key results
- On LINEMOD with K=64 reference views, achieves 78.4 ADD(S)-0.1d and 96.1 Proj2D average, outperforming OnePose++ (76.9 and 94.3).
- With K=16 on OnePose, reaches 87.8 at 5cm-5deg vs OnePose++ 82.8, and on OnePose-LowTexture 71.9 vs 51.6 at 5cm-5deg.
- CroCo pretraining gives 68.4 ADD(S)-0.1d on LINEMOD versus 39.4 with MAE and 16.6 without pretraining (K=16 ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MFOS instead of OnePose++ | One-shot pose estimation · LINEMOD | ADD(S)-0.1d: 76.9 -> 78.4 | Helped |
| use MFOS instead of OnePose++ | One-shot pose estimation · LINEMOD | Proj2D: 94.3 -> 96.1 | Helped |
| use MFOS instead of OnePose++ | One-shot pose estimation · OnePose dataset | 1cm-1deg: 51.1 -> 30.0 | Hurt |
| use MFOS instead of OnePose++ | One-shot pose estimation · OnePose-LowTexture | 5cm-5deg: 72.1 -> 74.2 | Helped |
| replace cuboid proxy shape with ellipsoid (ablation) | MFOS · LINEMOD | ADD(s): 60.9 -> 58.0 | Hurt |
| predict CAD-model coordinates instead of cuboid coordinates (ablation) | MFOS · LINEMOD | ADD(s): 60.9 -> 42.3 | Hurt |
| add OnePose to BOP training data (ablation) | MFOS training · LINEMOD | ADD(s): 44.2 -> 49.5 | Helped |
| add ABO to BOP + OnePose training data (ablation) | MFOS training · LINEMOD | ADD(s): 49.5 -> 60.9 | Helped |
| increase test reference views from K=16 to K=64 (ablation) | MFOS trained with K=16 · LINEMOD | ADD(s): 60.9 -> 65.3 | Helped |
| remove pretraining (ablation) | MFOS · LINEMOD | ADD(S)-0.1d: 68.4 -> 16.6 | Hurt |
| use MAE instead of CroCo pretraining (ablation) | MFOS · LINEMOD | ADD(S)-0.1d: 68.4 -> 39.4 | Hurt |
| perturb proxy-box rotation, translation, or scale (ablation) | MFOS · LINEMOD | ADD(S)-0.1d: 68.4 -> 68.2; 67.3; 69.4; 68.4; 67.3 | No clear effect |
Surprising findings
- Predicting coordinates on an invisible cuboid proxy shape works better than predicting coordinates on the actual CAD surface (cuboid-to-cuboid 60.9 vs cuboid-to-CAD 42.3 ADD(S)).
- The method is weaker than OnePose++ at the tightest 1cm-1deg threshold on the OnePose dataset (28.5 vs 51.1).
Limitations
- Requires reference images with pose annotations and approximate object size.
- Limited to rigid non-symmetrical objects; symmetric objects are excluded from training and fail in some cases.
- Requires roughly centered object crops, so it depends on a 2D object detector.
Experimental setup
Datasets: ABO BOP (T-LESS, HB, HOPE, YCB-V, RU-APC, TUD-L, TYO-L, ICMI) OnePose CroCo pretraining data
Benchmarks: LINEMOD OnePose OnePose-LowTexture ABO
Metrics: ADD(S)-0.1d Proj2D cm-degree accuracy median pose error
Baselines: Gen6D OnePose OnePose++ HLoc
Builds on: CroCo BB8 OnePose++ Vision Transformer
Closest papers by meaning
- Co-op: Correspondence-based Novel Object Pose Estimation (CVPR 2025 · similarity 0.88)
- NOPE: Novel Object Pose Estimation from a Single Image (CVPR 2024 · similarity 0.87)
- GIVEPose: Gradual Intra-class Variation Elimination for RGB-based Category-Level Object Pose Estimation (CVPR 2025 · similarity 0.86)
- GigaPose: Fast and Robust Novel Object Pose Estimation via One Correspondence (CVPR 2024 · similarity 0.86)
- Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation (ICLR 2024 · similarity 0.86)
- Free-Moving Object Reconstruction and Pose Estimation with Virtual Camera (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.