Research map / 3D vision / Gaussian splatting
HyperGS: Hyperspectral 3D Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperGS, a latent-space 3D Gaussian Splatting method with an autoencoder decoder for hyperspectral novel view synthesis, plus a new HNVS benchmark with adapted NeRF and 3DGS baselines.
Problem
Hyperspectral novel view synthesis is hard due to high-dimensional spectral data, unstable NeRF training, slow rendering, and naive 3DGS failing to model fine spectral detail.
Approach
Perform Gaussian splatting in the learned latent space of a frozen 1D convolutional spectral autoencoder (with SE blocks), using view-dependent hash-encoded MLP modulation, a gradient-depth-scaled densification score, a pixel-wise importance global pruning, and a Charbonnier plus cosine spectral loss with SSIM, initialized from SfM on grayscale band slices.
Key results
- On the BaySpec dataset, HyperGS reaches 27.11 PSNR, 0.7804 SSIM, 0.0254 SAM and 0.0440 RMSE, versus 26.53 PSNR for MipNeRF360 (best prior).
- On the SOP dataset, HyperGS reaches 30.51 PSNR and 0.00415 SAM, versus 28.58 PSNR for 3DGS.
- Abstract claims a 14 dB accuracy improvement over previously published models, though the tables show a smaller gain over the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use HyperGS instead of MipNeRF360 | HNVS · BaySpec datasets | PSNR: 27.11 vs 26.53 | Helped |
| Use HyperGS instead of 3DGS | HNVS · SOP datasets | PSNR: 30.51 vs 28.58 | Helped |
| Add spectral SFM initialization (ablation) | Base 3DGS · BaySpec datasets | PSNR: 22.91 -> 23.05 | Helped |
| Add latent-space autoencoder (ablation) | 3DGS with spectral SFM initialization · BaySpec datasets | PSNR: 23.05 -> 24.87 | Helped |
| Add latent hyperspectral densification (ablation) | 3DGS with latent-space AE · BaySpec datasets | PSNR: 24.87 -> 25.25 | Helped |
| Add global hyperspectral pruning (ablation) | 3DGS with densification · BaySpec datasets | PSNR: 25.25 -> 25.17 | Hurt |
| Add view-dependent MLP (ablation) | 3DGS with pruning · BaySpec datasets | PSNR: 25.17 -> 27.05 | Helped |
| Add custom loss function (ablation) | 3DGS with view-dependent MLP · BaySpec datasets | PSNR: 27.05 -> 27.11 | Helped |
Surprising findings
- Naive 3DGS performs worse than NeRF baselines on BaySpec (22.91 PSNR, SAM 0.1335), while MipNeRF360 is the strongest NeRF variant there.
- NeRF-style methods underperform 3DGS on the SOP dataset, which has far fewer training views (~40 per scene).
Limitations
- Naive 3DGS adaptation struggles with fine spectral detail, and densification can produce excessive Gaussians in high-frequency scenes.
- Autoencoder latent space bounds reconstruction error, but decoder quality limits final output; authors plan richer latent decoding as future work.
- Evaluation limited to two real HS-NeRF datasets; the ScanNet-based synthetic dataset is only in the supplement.
Experimental setup
Datasets: HS-NeRF BaySpec dataset HS-NeRF SOP dataset ScanNetv200 (synthetic hyperspectral variant, supplementary)
Benchmarks: HS-NeRF BaySpec dataset HS-NeRF SOP dataset
Metrics: PSNR SSIM SAM RMSE
Baselines: NeRF MipNeRF MipNeRF360 TensoRF (Spec-NeRF) Nerfacto HS-NeRF 3DGS
Builds on: 3D Gaussian Splatting Mip-Splatting Instant-NGP Mip-NeRF 360 HS-NeRF
Closest papers by meaning
- IBGS: Image-Based Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
- 3D-HGS: 3D Half-Gaussian Splatting (CVPR 2025 · similarity 0.90)
- DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting (CVPR 2025 · similarity 0.89)
- LP-3DGS: Learning to Prune 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.89)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.89)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.