Research map / 3D vision / Gaussian splatting
PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
PanSplat is a feed-forward, generalizable 3D Gaussian Splatting method for wide-baseline panorama novel view synthesis that scales to 4K resolution on a single A100 using a spherical Gaussian pyramid on a Fibonacci lattice and two-step deferred backpropagation.
Problem
Wide-baseline panorama novel view synthesis at high resolution (up to 4K) is limited by memory and compute costs, and pixel-aligned 3DGS produces redundant Gaussians near the poles of equirectangular panoramas.
Approach
Distribute Gaussians on a Fibonacci lattice over the sphere instead of pixel-aligned placement, predict them at multiple pyramid levels from a hierarchical spherical cost volume, and decouple geometry (low-res cost volume) from appearance (full-res Gaussian heads) with a two-step deferred backpropagation to train at 4K.
Key results
- On Matterport3D at 1.0m baseline, PanSplat reaches 28.81 WS-PSNR and 0.091 LPIPS, above MVSplat (28.19 / 0.105) and PanoGRF (27.12 / 0.195).
- PanSplat runs in 0.34 s per panorama versus 23.8 s for PanoGRF, which the paper describes as up to 70x faster.
- Fibonacci Gaussians cut the Gaussian count by up to 36.34% (1,049K to 668K) without hurting quality; the full model with the 3D Gaussian pyramid uses 887K Gaussians.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PanSplat instead of MVSplat | Novel view synthesis · Matterport3D, 1.0m baseline | WS-PSNR: 28.19 -> 28.81 | Helped |
| Use PanSplat instead of PanoGRF | Novel view synthesis · Matterport3D, 1.5m baseline | WS-PSNR: 23.38 -> 24.09 | Helped |
| Use PanSplat instead of PanoGRF | Novel view synthesis · Matterport3D, 2.0m baseline | WS-PSNR: 20.96 -> 20.56 | Hurt |
| Use PanSplat instead of PanoGRF | Novel view synthesis · Replica, 1.0m baseline | WS-PSNR: 29.22 -> 30.78 | Helped |
| Use PanSplat instead of PanoGRF | Novel view synthesis · Residential, about 0.3m baseline | WS-PSNR: 31.03 -> 30.97 | Hurt |
| Use PanSplat instead of MVSplat | Novel view synthesis · 360Loc | WS-PSNR: 24.67 -> 25.58 | Helped |
| Use PanSplat instead of MVSplat | Novel view synthesis · Insta360, 16 frames apart | WS-PSNR: 23.24 -> 24.43 | Helped |
| Add Fibonacci Gaussians (ablation) | PanSplat base model · Matterport3D, 1.0m baseline | WS-PSNR: 27.07 -> 27.86 | Helped |
| Add Fibonacci Gaussians (ablation) | PanSplat base model · Matterport3D, 1.0m baseline | #Gaussians (K): 1,049 (100%) -> 668 (63.67%) | Helped |
| Add 3D Gaussian pyramid (ablation) | PanSplat with Fibonacci Gaussians · Matterport3D, 1.0m baseline | WS-PSNR: 27.86 -> 28.81 | Helped |
| Add 3D Gaussian pyramid (ablation) | PanSplat with Fibonacci Gaussians · Matterport3D, 1.0m baseline | #Gaussians (K): 668 (63.67%) -> 887 (84.55%) | Hurt |
| Add deferred blending | PanSplat · 360Loc | WS-PSNR: 25.58 -> 28.14 | Helped |
Surprising findings
- MVSplat runs out of memory at 512x1024 training resolution in the memory comparison, while PanSplat handles 768x1536 and 4K on one A100.
Limitations
- Does not support dynamic scenes with moving objects.
- Real-world results rely on fine-tuning on 360Loc and a self-captured dataset.
- The finest Gaussian level uses full-resolution features while the cost volume is refined at lower resolution, and the finest cost volume refinement is skipped to save memory.
Experimental setup
Datasets: Matterport3D Replica Residential 360Loc Insta360 (self-captured)
Benchmarks: Matterport3D Replica Residential 360Loc Insta360
Metrics: WS-PSNR PSNR SSIM LPIPS
Baselines: S-NeRF OmniSyn IBRNet NeuRay PanoGRF MVSplat
Builds on: MVSplat PanoGRF pixelSplat 3D Gaussian Splatting Feature Pyramid Network
Closest papers by meaning
- Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.90)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.89)
- CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction (NeurIPS 2026 · similarity 0.89)
- MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.89)
- SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting (CVPR 2025 · similarity 0.88)
- 4K4DGen: Panoramic 4D Generation at 4K Resolution (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.