Research map / 3D vision / Gaussian splatting
OmniSplat: Taming Feed-Forward 3D Gaussian Splatting for Omnidirectional Images with Editable Capabilities
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces OmniSplat, a training-free feed-forward 3DGS method that adapts a perspective-trained MVSplat to omnidirectional images by decomposing them into Yin-Yang grid images and rasterizing Yin-Yang views.
Problem
Feed-forward 3D Gaussian splatting models are designed for perspective images, and equirectangular omnidirectional images have non-uniform sampling and distortion that break both feature encoders and novel-view rendering near the poles.
Approach
Decompose each equirectangular image into two quasi-uniform Yin-Yang square images so they resemble perspective images, run a pretrained MVSplat encoder with Yin-Yang cross-view attention, and render with a Yin-Yang rasterizer that combines the two rendered grids into an omnidirectional image.
Key results
- On OmniBlender, OmniSplat reaches 21.22 PSNR at 0.532 s runtime versus 17.48 for MVSplat (P) and 22.23 for ODGS at 1920 s.
- On OmniScenes, OmniSplat achieves 21.59 PSNR, 0.7649 SSIM, 0.3386 LPIPS, outperforming the best feed-forward baselines.
- OmniSplat+opt with a short color/opacity optimization reaches 23.12 PSNR on OmniScenes and beats ODGS on several datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OmniSplat instead of ODGS | Omnidirectional novel-view synthesis · OmniBlender | PSNR: 22.23 -> 21.22 | Hurt |
| Use OmniSplat instead of PixelSplat (P) | Omnidirectional novel-view synthesis · Ricoh360 | PSNR: 19.36 -> 19.72 | Helped |
| Use OmniSplat instead of ODGS | Omnidirectional novel-view synthesis · OmniPhotos | PSNR: 20.25 -> 18.59 | Hurt |
| Use OmniSplat instead of ODGS | Omnidirectional novel-view synthesis · 360Roam | PSNR: 18.72 -> 17.53 | Hurt |
| Use OmniSplat instead of ODGS | Omnidirectional novel-view synthesis · OmniScenes | PSNR: 20.71 -> 21.59 | Helped |
| Use OmniSplat instead of ODGS | Omnidirectional novel-view synthesis · 360VO | PSNR: 22.66 -> 20.63 | Hurt |
| Add color and opacity optimization on reference views while keeping Gaussian positions fixed | OmniSplat · OmniBlender, Ricoh360, OmniPhotos, 360Roam, OmniScenes, 360VO | PSNR: 21.22 -> 22.47; 19.72 -> 20.63; 18.59 -> 19.57; 17.53 -> 18.19; 21.59 -> 23.12; 20.63 -> 21.42 | Helped |
| Replace Yin-Yang rasterization with omnidirectional rasterization (ablation) | OmniSplat · OmniPhotos | PSNR: 18.59 -> 13.30 | Hurt |
| Replace Yin-Yang encoder decomposition with omnidirectional images (ablation) | OmniSplat · OmniPhotos | PSNR: 18.59 -> 15.97 | Hurt |
| Replace Yin-Yang rasterization with omnidirectional rasterization (ablation) | OmniSplat · OmniScenes | PSNR: 21.44 -> 16.59 | Hurt |
| Replace Yin-Yang encoder decomposition with omnidirectional images (ablation) | OmniSplat · OmniScenes | PSNR: 21.44 -> 17.82 | Hurt |
Surprising findings
- Naively feeding Yin-Yang images to MVSplat without the proposed cross-view attention and rasterizer performs worse than the omnidirectional baselines on some datasets.
- The optimization-based ODGS achieves the best reference-view quality but produces broken novel views due to overfitting with only two reference images.
Limitations
- Converting Yang images to the omnidirectional domain requires horizontal interpolation, reducing image quality.
- Requires a few optimization steps (OmniSplat+opt) to reach best results; adaptation steps could be further reduced.
- Relies on OpenMVG for pose estimation since the datasets lack poses.
Experimental setup
Datasets: OmniBlender Ricoh360 OmniPhotos 360Roam OmniScenes 360VO
Benchmarks: OmniBlender Ricoh360 OmniPhotos 360Roam OmniScenes 360VO
Metrics: PSNR SSIM LPIPS mIoU runtime
Baselines: PanoGRF PixelSplat LatentSplat MVSplat DepthSplat ODGS
Builds on: MVSplat pixelSplat ODGS Yin-Yang grid
Closest papers by meaning
- ODGS: 3D Scene Reconstruction from Omnidirectional Images with 3D Gaussian Splattings (NeurIPS 2024 · similarity 0.91)
- SC-OmniGS: Self-Calibrating Omnidirectional Gaussian Splatting (ICLR 2025 · similarity 0.89)
- Splatter Image: Ultra-Fast Single-View 3D Reconstruction (CVPR 2024 · similarity 0.88)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.88)
- SfM-Free 3D Gaussian Splatting via Hierarchical Training (CVPR 2025 · similarity 0.88)
- CodecSplat: Ultra-Compact Latent Coding for Feed-Forward 3D Gaussian Splatting (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.