Research map / 3D vision / Human pose estimation
NOPE: Novel Object Pose Estimation from a Single Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Predicts discriminative embeddings of novel views from a single reference image with a pose-conditioned U-Net, then template-matches a query to estimate relative object pose and its ambiguities without 3D models or per-object training.
Problem
Estimating the relative 3D pose of a novel object from only a single reference image, without a 3D model or retraining for new objects or categories, including handling pose ambiguities from symmetries and occlusions.
Approach
Instead of synthesizing pixel images of novel views (which invents unseen parts), train a pose-conditioned U-Net with cross-attention to directly predict the average embedding (in a frozen VAE latent space) of views under many relative poses; match query embeddings against these predicted templates, and read pose ambiguity from the matching distribution.
Key results
- On ShapeNet novel categories, the top-1 method reaches a mean Acc30 of 59.8 vs 48.8 for PIZZA and 46.1 for 3DiM; top-5 reaches 87.1.
- On T-LESS (single reference image, front views), VSD recall is 48.96 vs 19.19 for PIZZA, and it outperforms MultiPath (43.24).
- Under 25% occlusion, Acc30 is 40.5 vs 14.6 for PIZZA; one prediction takes 1.01 s to generate templates plus 0.18 s neighbor search, versus 13 min for 3DiM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use NOPE instead of PIZZA | single-image pose estimation · ShapeNet novel categories | Acc 30: 48.8 -> 59.8 | Helped |
| retrieve top 3 neighbors instead of top 1 | NOPE · ShapeNet novel categories | Acc 30: 59.8 -> 80.8 | Helped |
| retrieve top 5 neighbors instead of top 1 | NOPE · ShapeNet novel categories | Median pose error: 24.4 -> 6.6 | Helped |
| use NOPE instead of PIZZA | single-image pose estimation · T-LESS average | Recall VSD: 18.39 -> 46.49 | Helped |
| sample reference images from front views rather than all views | NOPE · T-LESS average | Recall VSD: 46.49 -> 48.96 | Helped |
| add rectangle occlusion covering 25% of the object bounding box | NOPE · ShapeNet | Acc 30: 59.8 -> 40.5 | Hurt |
| add rectangle occlusion covering 25% of the object bounding box | PIZZA · ShapeNet | Acc 30: 48.9 -> 14.6 | Hurt |
| predict novel-view embeddings in one step instead of using diffusion-based pixel synthesis | NOPE versus 3DiM · Novel view generation | Processing time: 13 min -> 1.01 s | Helped |
Surprising findings
- Predicting averaged embeddings beats pixel-level novel view synthesis (3DiM) for pose estimation, as diffusion models invent wrong details that hurt template matching.
- Novel-view-synthesis-based methods that regress pose from a conditioned feed-forward network generalize poorly to unseen categories.
Limitations
- Performance is poor on nearly symmetric objects (clock, dishwasher) and thin or barely visible objects (guitar); circular symmetry (mug) is not resolved by top-k neighbors.
- Only the PIZZA code was released; other baselines were re-implemented by the authors.
- Evaluation on synthetic ShapeNet renderings and T-LESS only; real-world category generalization is limited.
Experimental setup
Datasets: ShapeNet FORGE-style synthetic renderings T-LESS
Benchmarks: ShapeNet (novel instances and novel categories) T-LESS ShapeNet with partial occlusions
Metrics: Acc30 median pose error VSD recall run-time
Baselines: PIZZA SSVE ViewNet 3DiM MultiPath Nguyen et al. [32]
Builds on: 3DiM FORGE Nguyen et al. template matching (LINEMOD-style embedding matching) Stable Diffusion VAE Random Erasing
Closest papers by meaning
- 3D-Aware Hypothesis & Verification for Generalizable Relative Object Pose Estimation (ICLR 2024 · similarity 0.90)
- Source-Free and Image-Only Unsupervised Domain Adaptation for Category Level Object Pose Estimation (ICLR 2024 · similarity 0.89)
- UNOPose: Unseen Object Pose Estimation with an Unposed RGB-D Reference Image (CVPR 2025 · similarity 0.89)
- Unsupervised Learning of Category-Level 3D Pose from Object-Centric Videos (CVPR 2024 · similarity 0.89)
- One2Any: One-Reference 6D Pose Estimation for Any Object (CVPR 2025 · similarity 0.88)
- Dual Pose-invariant Embeddings: Learning Category and Object-specific Discriminative Representations for Recognition and Retrieval (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.