Research map / 3D vision / Gaussian splatting
IBGS: Image-Based Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Image-Based Gaussian Splatting (IBGS), which adds a learned color residual, predicted from warped pixel colors of neighboring training images, on top of the base 3DGS color to capture high-frequency detail and view-dependent effects without extra storage.
Problem
3D Gaussian Splatting uses low-degree spherical harmonics, so it struggles to capture high-frequency spatially varying color and view-dependent effects (e.g., specular highlights) without many Gaussians, while prior texture-based fixes either fail on complex scenes or add large storage overhead.
Approach
Model each pixel color as the 3DGS base color plus a residual predicted by a lightweight CNN from colors warped from nearby training images at median Gaussian-ray intersections, together with a neighbor-view exposure correction that generalizes to novel views.
Key results
- On Mip-NeRF 360, IBGS reaches 28.33 PSNR vs. 27.69 for 3DGS, using 1.59M Gaussians and 291 MB vs. 3.22M and 764 MB.
- On Tanks and Temples, IBGS reaches 24.84 PSNR and 0.869 SSIM, improving over TexturedGaussians' 24.26 PSNR.
- On three Shiny scenes, IBGS achieves at least 5.22 dB PSNR gain over SuperGaussians on Guitars and similar gains on Lab and CD, with fewer Gaussians.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use IBGS instead of the strongest listed baseline | IBGS · Mip-NeRF 360 | PSNR: 27.69 -> 28.33 | Helped |
| use IBGS instead of the strongest listed baseline | IBGS · Tanks and Temples (TNT) | PSNR: 24.28 -> 24.84 | Helped |
| use IBGS instead of the strongest listed baseline | IBGS · Deep Blending | PSNR: 29.53 -> 30.12 | Helped |
| use IBGS instead of SuperGauss | IBGS · Shiny: Guitars, Lab, and CD | PSNR: +5.22 dB at least | Helped |
| add predicted color residuals to the base color (ablation) | IBGS · Tanks and Temples (TNT) | PSNR: 23.06 -> 24.84 | Helped |
| add predicted color residuals to the base color (ablation) | IBGS · Mip-NeRF 360 | PSNR: 27.08 -> 28.33 | Helped |
| add color consistency loss L_photo (ablation) | IBGS · Tanks and Temples (TNT) | PSNR: 24.70 -> 24.84 | Helped |
| add color consistency loss L_photo (ablation) | IBGS · Mip-NeRF 360 | PSNR: 28.31 -> 28.33 | Helped |
| use source-color differences from the base color rather than full warped source colors as network input (ablation) | IBGS · Tanks and Temples (TNT) | PSNR: 24.61 -> 24.84 | Helped |
| use source-color differences from the base color rather than full warped source colors as network input (ablation) | IBGS · Mip-NeRF 360 | PSNR: 28.21 -> 28.33 | Helped |
| add exposure correction (ablation) | IBGS · Tanks and Temples (TNT) | PSNR: 24.28 -> 24.84 | Helped |
| lower opacity pruning threshold from 0.05 to 0.005 (ablation) | IBGS · Mip-NeRF 360 | PSNR: 28.33 -> 28.42 | Mixed |
Surprising findings
- Base color alone scores lower but the residual module with fewer Gaussians still improves quality under a higher opacity pruning threshold (0.05), suggesting better surface alignment reduces the Gaussian count needed.
Limitations
- Struggles in sparse-view settings where dense pixel correspondences are hard to obtain.
- Lower rendering speed and higher runtime memory than 3DGS due to extra rendering computation.
Experimental setup
Datasets: Tanks and Temples Deep Blending Mip-NeRF 360 Shiny
Benchmarks: Mip-NeRF 360 Tanks and Temples Deep Blending Shiny
Metrics: PSNR SSIM LPIPS number of Gaussians storage memory (MB)
Baselines: 3DGS SuperGaussians TexturedGaussians Mip-NeRF 360 Instant-NGP
Builds on: 3D Gaussian Splatting 2D Gaussian Splatting IBRNet SuperGaussians
Closest papers by meaning
- Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.95)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.93)
- Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.93)
- UVGS: Reimagining Unstructured 3D Gaussian Splatting using UV Mapping (CVPR 2025 · similarity 0.93)
- 3D-HGS: 3D Half-Gaussian Splatting (CVPR 2025 · similarity 0.93)
- Anti-Aliased 2D Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.