Research map / 3D vision / Gaussian splatting
SGSST: Scaling Gaussian Splatting Style Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Optimizes only the base colors of a pretrained 3D Gaussian splatting scene with a parameterless multiscale VGG Gram/mean/std style loss (SOS) to achieve ultra-high-resolution 3D style transfer.
Problem
Style transfer of full 3D scenes represented by 3DGS has been limited to medium resolution outputs with blurry or weak texture transfer, failing to reproduce high-resolution multiscale painting textures.
Approach
A Simultaneously Optimized Scales (SOS) loss averages the SPST-style VGG19 transfer loss over several downscaled resolutions and optimizes it simultaneously, while only the Gaussians' constant (degree-0) color components are updated and all geometry is frozen, enabling UHR 3DGS style transfer via partition-based gradient computation.
Key results
- SGSST achieves the lowest Gram loss (2.59e8) vs StyleGaussian (4.61e8) and ARF (5.77e8) in the 40-experiment comparison.
- In a 680-vote perceptual study, SGSST was chosen as most style-faithful 66.3% of the time vs 31.6% for ARF and 2.1% for StyleGaussian.
- SGSST renders 3DGS style transfer at ultra-high resolution (5187x3361), taking 25.5 hours on a single A100 for the garden scene.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · 40 style-transfer experiments | Gram loss: SGSST: 2.59e8; StyleGaussian: 4.61e8; ARF: 5.77e8 | Helped |
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · 40 style-transfer experiments, short-range consistency | LPIPS: SGSST: 0.030; StyleGaussian: 0.033; ARF: 0.040 | Helped |
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · 40 style-transfer experiments, short-range consistency | RMSE: SGSST: 0.032; StyleGaussian: 0.029; ARF: 0.037 | Mixed |
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · 40 style-transfer experiments, long-range consistency | LPIPS: SGSST: 0.055; StyleGaussian: 0.050; ARF: 0.072 | Mixed |
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · 40 style-transfer experiments, long-range consistency | RMSE: SGSST: 0.063; StyleGaussian: 0.056; ARF: 0.066 | Mixed |
| Use SGSST instead of ARF or StyleGaussian | 3DGS style transfer · Perceptual study | vote share: SGSST: 66.3%; ARF: 31.6%; StyleGaussian: 2.1% | Helped |
| Optimize all spherical-harmonic coefficients instead of only the constant color components (ablation) | SGSST · 3DGS style-transfer examples | stylization quality | No clear effect |
| Optimize all 3DGS parameters instead of only the constant color components (ablation) | SGSST · 3DGS style-transfer examples | geometry preservation | Hurt |
| Replace simultaneous multiscale optimization with coarse-to-fine optimization (ablation) | SGSST · 3DGS style transfer | large-scale style features | Hurt |
| Remove the initial 10k-iteration coarsest-scale color-transfer step (ablation) | SGSST · 3DGS style transfer | style color-palette reproduction | Hurt |
| Use fewer style-transfer scales instead of combining all available scales (ablation) | SGSST · UHR 3DGS style transfer | style-transfer quality | Hurt |
Surprising findings
- Coarse-to-fine multiscale strategy fails for 3DGS: gains at a scale are lost when optimizing the next finer scale, unlike for 2D pixel-grid style transfer.
- Optimizing additional 3DGS parameters (spherical harmonics or all parameters) does not improve stylization and can severely degrade geometry.
Limitations
- Large computation time, from minutes to hours, far slower than feed-forward 3DGS stylization methods
- Gram-based global statistics cause color bleeding or texturing of flat areas when content and style strongly mismatch
- Limited texture representation in low-density Gaussian regions causes isolated Gaussians to be visible
Experimental setup
Datasets: 3DGS multiview scenes (garden, train, truck, counter) style painting images
Benchmarks: 40 3D style transfer experiments over 9 scenes
Metrics: Gram loss LPIPS short-range and long-range consistency RMSE short-range and long-range consistency perceptual user preference votes
Baselines: ARF StyleGaussian StylizedGS G-Style
Builds on: 3D Gaussian splatting SPST Gatys neural style transfer ARF
Closest papers by meaning
- CLIPGaussian: Universal and Multimodal Style Transfer Based on Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
- Multi-StyleGS: Stylized Gaussian Splatting with Multiple Styles (AAAI 2025 · similarity 0.91)
- ReGS: Reference-based Controllable Scene Stylization with Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS (NeurIPS 2025 · similarity 0.90)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.89)
- Perceptual-GS: Scene-adaptive Perceptual Densification for Gaussian Splatting (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.