Research map / 3D vision / Human pose estimation
PanoPose: Self-supervised Relative Pose Estimation for Panoramic Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PanoPose, a fully self-supervised depth-net + transformer pose-net for scaled relative pose of panoramic images, with rotation-only pre-training and a depth fusion block, plus an L2IRLS translation averaging global SfM pipeline.
Problem
Scaled relative pose estimation between panoramic images for global SfM, where two-view translation from the five-point algorithm is scaleless and pose estimation degrades under large viewing-angle differences and textureless scenes.
Approach
A self-supervised photometric-reprojection network with a CroCo transformer pose-net that regresses 6-DoF relative pose, fused with depth features via a transformer fusion block to recover translation scale, pre-trained with random panoramic rotations to handle large viewing-angle differences; paired with a translation averaging method (L2IRLS) that directly optimizes relative translation scales.
Key results
- On 360VO-Seq1 and PanoSUNCG, PanoPose reduces mean RRE, median RRE, mean RTAE and median RTAE by 87%, 61%, 81% and 36% versus the best self-supervised baseline.
- Using PanoPose with L2IRLS instead of five-point with BATA reduces mean and median ATE by 29% and 27% on average across datasets.
- Rotation-only pre-training cuts RRE by about 60%; the fusion block reduces RSE by 22%, 44%, and 73% on the three ablation datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PanoPose instead of five-point estimation | relative pose estimation · 360VO-Seq1 | Mean RSE: 1.8622 -> 0.1630 | Helped |
| Use PanoPose instead of five-point estimation | relative pose estimation · PanoSUNCG | Mean RRE: 1.2548 -> 0.1559 | Helped |
| Use PanoPose instead of five-point estimation | relative pose estimation · Mapillary Metropolis | Mean RSE: 0.0133 -> 0.0217 | Hurt |
| Use PanoPose instead of five-point relative poses | global SfM with L2IRLS · 360VO-Seq3 | Mean ATE: 3.1602 -> 1.9853 | Helped |
| Add rotation-only pre-training (ablation) | PanoPose · 360VO-Seq4 | RRE: 0.1795 -> 0.0687 | Helped |
| Add rotation-only pre-training (ablation) | PanoPose · PanoSUNCG | RRE: 0.8459 -> 0.2237 | Helped |
| Add rotation-only pre-training (ablation) | PanoPose · Campus | RRE: 0.2476 -> 0.1404 | Helped |
| Add rotation-only pre-training (ablation) | PanoPose · Campus | RTAE: 2.6856 -> 3.1764 | Hurt |
| Add depth fusion block (ablation) | PanoPose · 360VO-Seq4 | RSE: 0.2317 -> 0.1806 | Helped |
| Add depth fusion block (ablation) | PanoPose · PanoSUNCG | RSE: 3.0462 -> 1.6933 | Helped |
| Add depth fusion block (ablation) | PanoPose · Campus | RSE: 0.8419 -> 0.2314 | Helped |
| Add depth fusion block (ablation) | PanoPose · Campus | RTAE: 2.6856 -> 2.8506 | Hurt |
Surprising findings
- The traditional five-point algorithm with RootSIFT matching outperforms PanoPose on relative rotation and translation direction in most datasets.
- Pairing the five-point algorithm with L2IRLS yields the highest ATE, since unknown initial scale degrades scale optimization.
Limitations
- Transformer-based CroCo backbone is computationally heavy, affecting training and inference efficiency.
- Cross-dataset generalization is limited.
- Five-point algorithm outperforms PanoPose on several rotation and translation-direction metrics.
Experimental setup
Datasets: PanoSUNCG Mapillary Metropolis 360VO Building (self-collected) Campus (self-collected)
Benchmarks: PanoSUNCG Mapillary Metropolis 360VO Building Campus
Metrics: relative rotation error (RRE) relative translation angle error (RTAE) relative scale error (RSE) absolute rotation error (ARE) absolute translation error (ATE)
Baselines: SfMLearner MonoDepth2 NonLocal-DPT BiFuse++ Five-point algorithm BATA LUD
Builds on: Monodepth2 SfMLearner CroCo BATA Chatterjee and Govindu rotation averaging
Closest papers by meaning
- FAR: Flexible Accurate and Robust 6DoF Relative Camera Pose Estimation (CVPR 2024 · similarity 0.86)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.85)
- Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.85)
- PanoWorld: Geometry-Consistent Panoramic Video World Modeling (NeurIPS 2026 · similarity 0.84)
- Relative Pose Estimation through Affine Corrections of Monocular Depth Priors (CVPR 2025 · similarity 0.84)
- Learning to Filter Outlier Edges in Global SfM (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.