Research map / 3D vision / Human pose estimation
Sparse-view Pose Estimation and Reconstruction via Analysis by Generative Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SparseAGS, which jointly refines camera poses and 3D (via DreamGaussian-style multi-view SDS with a 6-DoF Zero-1-to-3 prior) from off-the-shelf pose estimates, with iterative outlier removal and discrete-plus-continuous pose correction.
Problem
Joint camera pose estimation and 3D reconstruction from sparse unposed object-centric views, where initial pose estimates may contain large errors and sparse observations under-constrain 3D.
Approach
Extends analysis-by-synthesis with a finetuned 6-DoF novel-view diffusion prior (Zero-1-to-3 with 18-dim 6-DoF camera conditioning) used via multi-view SDS, plus iterative outlier detection (removing images that degrade reconstruction of others) and a discrete sphere-sampling pose search with LPIPS/MSE ranking to correct outlier poses.
Key results
- With DUSt3R initialization on NAVI at N=8, the method reaches 83.7% Rot.@5° (vs 52.3% for DUSt3R alone) and 93.5% CC@0.1 (vs 82.2%).
- On NAVI with 8 views and Ray Diffusion initialization, PSNR improves from 13.44 (no outlier handling) to 15.30 and F1@0.01 from 57.2 to 68.2 after outlier removal and correction.
- For 3D reconstruction on NAVI with N=10, the method reaches 18.03 PSNR / 0.1660 LPIPS with DUSt3R init, versus LEAP 12.98 / 0.2890.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| refine initial pose estimates with SparseAGS | RelPose++ · NAVI, 8 images | Rot.@5°: 10.9 -> 42.1 | Helped |
| refine initial pose estimates with SparseAGS | Ray Diff. · NAVI, 8 images | CC@0.1: 38.3 -> 80.4 | Helped |
| refine initial pose estimates with SparseAGS | DUSt3R · NAVI, 8 images | Rot.@5°: 52.3 -> 83.7 | Helped |
| refine initial DUSt3R poses with SPARF | DUSt3R · NAVI, 8 images | Rot.@15°: 93.8 -> 87.8 | Hurt |
| refine initial poses with SparseAGS rather than pose-free reconstruction | LEAP · NAVI, 8 images | PSNR: 12.93 -> 17.03 | Helped |
| add generative SDS priors to pose-3D co-optimization (ablation) | Pose-3D co-optimization without SDS · NAVI, 8 images, Ray Diff. initialization | PSNR: 12.72 -> 13.44 | Helped |
| use vanilla Zero123 camera conditioning instead of no SDS (ablation) | Pose-3D co-optimization · NAVI, 8 images, Ray Diff. initialization | Rot.@15°: 79.9 -> 78.3 | Hurt |
| use 6-DoF rather than vanilla Zero123 camera conditioning (ablation) | SDS pose-3D co-optimization · NAVI, 8 images, Ray Diff. initialization | Rot.@5°: 30.2 -> 34.6 | Helped |
| add outlier removal and correction (ablation) | SDS with 6-DoF Zero123 · NAVI, 8 images, Ray Diff. initialization | Rot.@5°: 34.6 -> 60.3 | Helped |
| add outlier removal and correction (ablation) | SDS with 6-DoF Zero123 · NAVI, 8 images, Ray Diff. initialization | PSNR: 13.44 -> 15.30 | Helped |
| add outlier removal and correction (ablation) | SDS with 6-DoF Zero123 · NAVI, 8 images, Ray Diff. initialization | F1@0.01: 57.2 -> 68.2 | Helped |
Surprising findings
- SPARF's correspondence-based pose refinement degraded accuracy relative to some initializations (e.g., Rot.@15° dropped for RelPose++ and DUSt3R), attributed to false correspondences.
- Vanilla Zero-1-to-3 as a generative prior reduced camera rotation and center accuracy at 15° and CC@0.1 compared to no prior, due to its 3-DoF camera parameterization.
Limitations
- Requires reasonable off-the-shelf pose estimates; fails if a large fraction of initial predictions have large error.
- Does not handle truncation or occlusion, so close-up partial views or cluttered scenes are unsupported.
- Evaluated only in an object-centric setting.
Experimental setup
Datasets: NAVI CO3D Objaverse GSO ABO OmniObject3D
Benchmarks: NAVI GSO ABO OmniObject3D
Metrics: Rotation accuracy@5° Rotation accuracy@15° Camera center accuracy@0.1 PSNR LPIPS F1@0.01
Baselines: RelPose++ Ray Diffusion DUSt3R SPARF LEAP UpFusion ID-Pose
Builds on: DreamGaussian Zero-1-to-3 SPARF Score Distillation Sampling
Closest papers by meaning
- LEAP: Liberate Sparse-View 3D Modeling from Camera Poses (ICLR 2024 · similarity 0.92)
- No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images (ICLR 2025 · similarity 0.91)
- Cameras as Rays: Pose Estimation via Ray Diffusion (ICLR 2024 · similarity 0.90)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.89)
- SPARS3R: Semantic Prior Alignment and Regularization for Sparse 3D Reconstruction (CVPR 2025 · similarity 0.89)
- SPAR3D: Stable Point-Aware Reconstruction of 3D Objects from Single Images (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.