Research map / 3D vision / Human pose estimation
SCFlow2: Plug-and-Play Object Pose Refiner with Shape-Constraint Scene Flow
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCFlow2, a plug-and-play RGBD 6D object pose refiner that predicts 3D scene flow (SE3 motion field) inside a recurrent matching network with shape-constraint pose-induced flow, trained on Objaverse, GSO and ShapeNet without retraining on novel objects.
Problem
Pose refinement for 6D object pose estimation of novel objects is either noisy in correspondence building, requires retraining per object, or uses 2D flow plus a separate RANSAC-Kabsch depth stage that is only locally optimal.
Approach
Replaces SCFlow's 2D optical-flow intermediate with a dense 3D scene flow (SE3 motion field with rigid-motion embeddings from RAFT-3D), fuses depth into the correlation volume, and pools the motion field into a global pose residual, keeping the mesh-rendered pose-induced flow as shape constraint so the refiner is end-to-end trainable and generalizes to novel objects.
Key results
- As a post-processing step on FoundationPose, average BOP AR rises from 73.4 to 75.2 across seven datasets (YCB-V 88.9 to 91.4, LM-O 75.6 to 80.1).
- On FoundPose initialization, SCFlow2 raises average AR from 59.6 to 68.6 without any retraining, exceeding the retrained SCFlow V1+ variant.
- Refinement with one pose hypothesis takes about 0.18 s per image on an RTX-3090, more than 10x faster than MegaPose (1.85 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SCFlow2 as a post-processing refiner | FoundationPose · LM-O | AR: 75.6 -> 80.1 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · T-LESS | AR: 64.6 -> 65.0 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · TUD-L | AR: 92.3 -> 95.5 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · IC-BIN | AR: 50.8 -> 51.7 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · ITODD | AR: 58.0 -> 58.3 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · HB | AR: 83.5 -> 84.4 | Helped |
| Apply SCFlow2 as a post-processing refiner | FoundationPose · YCB-V | AR: 88.9 -> 91.4 | Helped |
| Add shape-constraint regularization (ablation) | SCFlow2 · LM-O | AR: 60.0 -> 61.3 | Helped |
| Add shape-constraint regularization (ablation) | SCFlow2 · YCB-V | AR: 80.4 -> 81.8 | Helped |
| Use 3D scene flow instead of a plain MLP pose regressor (ablation) | SCFlow2 · LM-O | AR: 52.6 -> 61.3 | Helped |
| Use 3D scene flow instead of a plain MLP pose regressor (ablation) | SCFlow2 · YCB-V | AR: 67.6 -> 81.8 | Helped |
Surprising findings
- Removing the 3D scene flow formulation drops performance significantly (YCB-V 80.4 vs 81.8 with both), and a version without retraining but with RANSAC Kabsch depth (V1++) performs worse than the retrained V1+.
Limitations
- Refinement sometimes makes poses worse when the initial pose is very wrong due to severe occlusion or extreme lighting.
- Requires a good enough initialization; robustness degrades at high initial noise levels (L40-L50).
- Evaluated only on BOP datasets with pose-initialization from other methods.
Experimental setup
Datasets: Objaverse Google Scanned Objects ShapeNet-Objects BOP LM-O BOP T-LESS BOP TUD-L BOP IC-BIN BOP ITODD BOP HB BOP YCB-V
Benchmarks: BOP LM-O BOP T-LESS BOP TUD-L BOP IC-BIN BOP ITODD BOP HB BOP YCB-V
Metrics: BOP mean AR VSD MSSD MSPD
Baselines: MegaPose FoundPose GenFlow GigaPose SAM6D FoundationPose SCFlow
Builds on: SCFlow RAFT-3D RAFT DenseFusion
Closest papers by meaning
- GenFlow: Generalizable Recurrent Flow for 6D Pose Refinement of Novel Objects (CVPR 2024 · similarity 0.90)
- RefPose: Leveraging Reference Geometric Correspondences for Accurate 6D Pose Estimation of Unseen Objects (CVPR 2025 · similarity 0.87)
- GeoReF: Geometric Alignment Across Shape Variation for Category-level Object Pose Refinement (CVPR 2024 · similarity 0.86)
- SchemaPose: RGB-Based Category-Level Pose Estimation with Parametric Category Schema (NeurIPS 2026 · similarity 0.86)
- Rectified Point Flow: Generic Point Cloud Pose Estimation (NeurIPS 2025 · similarity 0.86)
- SingRef6D: Monocular Novel Object Pose Estimation with a Single RGB Reference (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.