Research map / 3D vision / Depth and stereo
OmniStereo: Real-time Omnidireactional Depth Estimation with Multiview Fisheye Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OmniStereo, a real-time omnidirectional depth estimator from multi-view fisheye cameras that combines Cassini-projection stereo matching, confidence-guided fusion, and depth refinement.
Problem
Omnidirectional depth estimation from multiple fisheye cameras is too slow for real-time use and has accuracy issues such as depth discontinuities across views, invalid pixels, and mapping-induced errors.
Approach
Rectify fisheye images to Cassini projections so epipolar lines become horizontal, run an ultra-lightweight stereo network (1.8M params) per direction, fuse the four depth maps with confidence-weighted softmax, then refine with a small local depth-hypothesis cost volume.
Key results
- On Urban, OmniStereo reaches MAE 6.57 and RMSE 60.26 versus 12.14 and 92.20 for OmniMVS+, a ~32% MAE and ~16.5% RMSE improvement over the second-best method.
- Runs at 35 FPS on TITAN RTX at 640x320 and 24.6 FPS at 1920x960; 12.3 FPS at 640x320 on Jetson AGX Orin.
- On unseen OmniTown, OmniStereo achieves MAE 13.28 versus 21.80 for MODE, the second-best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the strongest reported baseline with OmniStereo | OmniStereo vs. MODE · Urban | MAE: 16.24 -> 6.57 | Helped |
| Replace the strongest reported baseline with OmniStereo | OmniStereo vs. MODE · OmniHouse | MAE: 3.57 -> 1.76 | Helped |
| Replace the strongest reported baseline with OmniStereo | OmniStereo vs. CrownConv360 · OmniThings | MAE: 56.80 -> 31.66 | Helped |
| Add confidence-guided fusion to stereo matching (ablation) | OmniStereo on Urban · Urban | MAE: 21.46 -> 7.91 | Helped |
| Add depth refinement to the fusion method (ablation) | OmniStereo on Urban · Urban | MAE: 7.91 -> 6.57 | Helped |
| Replace the Fast-ACVNet+ baseline with the proposed stereo matching method without fusion or refinement (ablation) | OmniStereo on Urban · Urban | MAE: 15.78 -> 21.46 | Hurt |
| Replace the Fast-ACVNet+ baseline with the proposed stereo matching method without fusion or refinement (ablation) | OmniStereo on Urban · Urban | δ1 (%): 84.45 -> 86.73 | Helped |
Surprising findings
- Spherical sweeping methods achieve competitive delta thresholds and SILog but fall far behind on MAE and RMSE.
Limitations
- Accuracy gains are reported only against the stated baselines and the Urban/OmniHouse/OmniThings datasets with the authors' protocol.
- OmniVidar comparison is in the supplementary under different protocols, so not directly comparable in the main results.
Experimental setup
Datasets: Urban OmniHouse OmniThings OmniTown
Benchmarks: Urban OmniHouse OmniThings OmniTown
Metrics: MAE RMSE SILog delta1 delta2 delta3 FPS
Baselines: OmniMVS+ OmniMVS+Tiny CrownConv360 Sphere-Stereo Fast-OmniMVS MODE
Builds on: ACVNet CUBG MODE GwcNet GC-Net
Closest papers by meaning
- SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting (ICLR 2025 · similarity 0.85)
- HELVIPAD: A Real-World Dataset for Omnidirectional Stereo Depth Estimation (CVPR 2025 · similarity 0.85)
- ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings (NeurIPS 2024 · similarity 0.84)
- UniK3D: Universal Camera Monocular 3D Estimation (CVPR 2025 · similarity 0.84)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.83)
- OmniSDF: Scene Reconstruction using Omnidirectional Signed Distance Functions and Adaptive Binoctrees (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.