Research map / 3D vision / Gaussian splatting
SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SC-OmniGS, a self-calibrating omnidirectional Gaussian splatting system that jointly optimizes 360-degree camera poses, a learnable omnidirectional camera distortion model, and 3D Gaussians from noisy or no pose priors.
Problem
Radiance field reconstruction from 360-degree images is sensitive to camera intrinsic and extrinsic errors, since existing models assume an ideal spherical camera and self-calibration methods target perspective images.
Approach
Treat 360-degree images as a whole sphere: splat 3D Gaussians onto an equirectangular unit sphere, derive gradients w.r.t. omnidirectional camera poses (quaternion + translation), and learn a differentiable omnidirectional distortion model (spherical grid with angle distortion coefficients) that undistorts inputs, all optimized with a weighted spherical photometric loss and anisotropy regularizer.
Key results
- On 360Roam with perturbed poses and SfM init, SC-OmniGS reaches 24.910 test PSNR versus 15.619 for OmniGS under the same perturbation.
- On OmniBlender trained from scratch, SC-OmniGS (render depth init) reaches test PSNR of 34.368 on Barbershop, 30.212 on Classroom, and 32.424 on Flat, versus 17.120 to 18.758 for OmniGS from scratch.
- On 360Roam without perturbation, SC-OmniGS improves test PSNR over OmniGS (25.297 vs 24.212) with SfM initialization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add omnidirectional camera model (learned distortion) on top of baseline, no pose optimization (ablation) | SC-OmniGS on 360Roam Center · 360Roam Center | PSNR test (w/o perturb): 24.264 -> 25.123 | Helped |
| add pose optimization only (ablation) | SC-OmniGS on 360Roam Center · 360Roam Center w/ perturbation | PSNR test: 15.597 -> 24.739 | Helped |
| add omnidirectional camera model only (no pose optimization) (ablation) | SC-OmniGS on 360Roam Center · 360Roam Center w/ perturbation | PSNR test: 15.597 -> 15.702 | No clear effect |
| add pose optimization to camera model (ablation) | SC-OmniGS on 360Roam Center · 360Roam Center without perturbation | PSNR test: 25.123 -> 25.802 (camera model + pose vs camera model only); pose-only 24.906 is below camera model only | Mixed |
| add both camera model and pose optimization (ablation) | SC-OmniGS on 360Roam Center · 360Roam Center w/ perturbation | PSNR test: 15.597 -> 25.304 | Helped |
| remove Gaussian re-initialization at 2000th and 4000th iterations (ablation) | SC-OmniGS on OmniBlender Classroom (from scratch) · OmniBlender Classroom | PSNR test: 29.183 (w/o) vs 30.212 (full) | Helped |
| replace weighted spherical photometric loss with classical photometric loss (ablation) | SC-OmniGS on OmniBlender Classroom (from scratch) · OmniBlender Classroom | PSNR test: 28.225 (w/o L_wsp) vs 30.212 (full) | Helped |
| use SC-OmniGS with SfM init vs OmniGS (SfM, no perturb) on real data | 360Roam 8-scene average · 360Roam test | PSNR test: 24.212 -> 25.297 | Helped |
| apply omnidirectional sampling to NeRF baselines (BARF, L2G-NeRF, CamP) | BARF, L2G-NeRF, CamP on 360Roam with perturbation · 360Roam test | PSNR test: BARF 20.200 -> BARF° 20.484; L2G-NeRF 20.507 -> 20.023; CamP 14.253 -> 13.659 | Mixed |
| random initialization instead of SfM point cloud with perturbed poses | SC-OmniGS on 360Roam · 360Roam test | PSNR test: 24.910 (SfM) vs 24.343 (random) | No clear effect |
| train from scratch with no pose prior on 360Roam | All self-calibration methods · 360Roam (multi-room) | qualitative (fail to learn) | Hurt |
Surprising findings
- Naively converting 360-degree images to cube maps and running perspective self-calibration baselines often performs poorly, while the same baselines with omnidirectional ray sampling improve substantially.
- All self-calibration methods, including SC-OmniGS, fail to learn radiance fields from scratch without pose priors on multi-room 360Roam scenes.
Limitations
- Training from scratch fails on challenging multi-room scenes with sparse discrete views without an SfM pipeline.
- Performance degrades with increasing rotation noise, more sensitive to rotation than translation errors.
Experimental setup
Datasets: OmniBlender 360Roam
Benchmarks: OmniBlender 360Roam
Metrics: PSNR SSIM LPIPS camera position RMSE camera rotation RMSE
Baselines: 3D-GS OmniGS BARF L2G-NeRF CamP BARF (omnidirectional sampling) L2G-NeRF (omnidirectional sampling) CamP (omnidirectional sampling)
Builds on: 3D Gaussian splatting OmniGS BARF L2G-NeRF CamP
Closest papers by meaning
- ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings (NeurIPS 2024 · similarity 0.90)
- OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities (CVPR 2025 · similarity 0.89)
- OmniLocalRF: Omnidirectional Local Radiance Fields from Dynamic Videos (CVPR 2024 · similarity 0.88)
- 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes (CVPR 2025 · similarity 0.88)
- COLMAP-Free 3D Gaussian Splatting (CVPR 2024 · similarity 0.87)
- SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.