Research map / 3D vision / Depth and stereo
Deep Single Image Camera Calibration by Heatmap Regression to Recover Fisheye Images Under Manhattan World Assumption
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a heatmap-regression VP/ADP estimator plus a distortion estimator to recover pan, tilt, roll, focal length and distortion from a single fisheye image in a Manhattan world.
Problem
Single-image camera calibration (rotation and fisheye distortion) in a Manhattan world is unreliable for general scenes lacking lines, arcs, and vanishing points.
Approach
Treats VP estimation as heatmap-based keypoint detection (like pose estimation) and adds eight auxiliary diagonal points (ADPs) arranged with octahedral symmetry to supply extra geometric constraints when VPs are absent; rotation is recovered by absolute orientation fitting.
Key results
- On SL-MH, pan error of 2.20 degrees versus 22.36 for Lochman et al., with tilt 3.15 and roll 3.00 degrees, the best among compared methods.
- Adding ADPs reduced average angle error by 15.4 degrees compared with VP-only (5 points).
- The VP estimator runs at about 12.3 fps (HRNet-W32), at least 280 times faster than the geometry-based pan-angle methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use heatmap-regressed VP/ADPs with HRNet-W32 instead of Lochman et al.'s geometry-based method | camera calibration · SL-MH test | pan error: 22.36 -> 2.20 | Helped |
| use heatmap-regressed VP/ADPs with HRNet-W32 instead of Lochman et al.'s geometry-based method | camera calibration · SL-MH test | tilt error: 44.42 -> 3.15 | Helped |
| use heatmap-regressed VP/ADPs with HRNet-W32 instead of Lochman et al.'s geometry-based method | camera calibration · SL-MH test | roll error: 33.20 -> 3.00 | Helped |
| use heatmap-regressed VP/ADPs with HRNet-W32 instead of Lochman et al.'s geometry-based method | camera calibration · SL-PB, SP360, and HoliCity tests | pan, tilt, and roll errors | Helped |
| add ADPs to the five VPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | pan error: 19.38 -> 2.20 | Helped |
| add ADPs to the five VPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | tilt error: 13.54 -> 3.15 | Helped |
| add ADPs to the five VPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | roll error: 21.65 -> 3.00 | Helped |
| retain VPs alongside the eight ADPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | pan error: 10.54 -> 2.20 | Helped |
| retain VPs alongside the eight ADPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | tilt error: 11.01 -> 3.15 | Helped |
| retain VPs alongside the eight ADPs (ablation) | camera calibration with HRNet-W32 · SL-MH test | roll error: 8.11 -> 3.00 | Helped |
Surprising findings
- Geometry-based Lochman et al. method executed on only 59.1% of images, showing dependence on arcs.
Limitations
- Fails when an image has one or no unique axes identifiable from VP/ADPs (over 98% coverage only if all points detected).
- Only single-image input is considered; multi-image or video not explored.
- Back labels ignored due to rarity, and pan angle range limited to -90 to 90 degrees because of front/back ambiguity.
Experimental setup
Datasets: StreetLearn (SL-MH, SL-PB) SP360 HoliCity ImageNet
Benchmarks: SL-MH test set SL-PB test set SP360 test set HoliCity test set off-the-shelf fisheye cameras
Metrics: mean absolute angle error (pan, tilt, roll) reprojection error (REPE) executable rate AP AR PCK fps
Baselines: López-Antequera et al. (CVPR'19) Wakai and Yamashita (ICCVW'21) Wakai et al. (ECCV'22) Pritts et al. (CVPR'18) Lochman et al. (WACV'21)
Builds on: HRNet Wakai et al. generic camera calibration network Manhattan world assumption DARK
Closest papers by meaning
- DirectFisheye-GS: Enabling Native Fisheye Input in Gaussian Splatting with Cross-View Joint Optimization (CVPR 2026 · similarity 0.84)
- FisheyeHDK: Hyperbolic Deformable Kernel Learning for Ultra-Wide Field-of-View Image Recognition (AAAI 2022 · similarity 0.84)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.84)
- AnyMap: Learning a General Camera Model for Structure-from-Motion with Unknown Distortion in Dynamic Scenes (CVPR 2025 · similarity 0.83)
- Reloc3r: Large-Scale Training of Relative Camera Pose Regression for Generalizable, Fast, and Accurate Visual Localization (CVPR 2025 · similarity 0.83)
- UniK3D: Universal Camera Monocular 3D Estimation (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.