Research map / 3D vision / Gaussian splatting
HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HiSplat, a coarse-to-fine hierarchical generalizable 3D Gaussian Splatting framework that predicts multi-scale pixel-aligned Gaussians from two views, linked by an Error Aware Module and a Modulating Fusion Module.
Problem
Generalizable feed-forward 3D Gaussian Splatting methods predict single-scale per-pixel Gaussians from sparse views, which struggle to capture both large-scale structure and fine texture details, causing mislocation and artefacts.
Approach
Generate coarse-grained Gaussians first for large-scale structure, then fine-grained Gaussians placed near them; an error-aware module predicts bounded depth offsets from rendering error maps, and a modulating fusion module reweights earlier-stage Gaussian opacities, so the stages jointly optimize with supervision at every stage.
Key results
- On RealEstate10K, HiSplat reaches 27.21 PSNR, +0.82 over MVSplat (26.39) and +0.52 over TranSplat (26.69).
- On ACID, HiSplat reaches 28.75 PSNR, versus 28.35 for TranSplat and 28.25 for MVSplat.
- Zero-shot from RealEstate10K, HiSplat gains +3.19 PSNR on Replica (27.17 vs 23.98 for PixelSplat) and +1.05 PSNR on DTU over the best prior baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single-scale Gaussians with full hierarchical HiSplat | generalizable 3D Gaussian Splatting · RealEstate10K | PSNR: +0.82 PSNR | Helped |
| replace single-scale Gaussians with full hierarchical HiSplat | generalizable 3D Gaussian Splatting · ACID | PSNR: +0.5 PSNR | Helped |
| replace single-scale Gaussians with full hierarchical HiSplat | models trained on RealEstate10K · RealEstate10K → DTU (zero-shot) | PSNR: +1.05 PSNR | Helped |
| replace single-scale Gaussians with full hierarchical HiSplat | models trained on RealEstate10K · RealEstate10K → ACID (zero-shot) | PSNR: +0.55 PSNR | Helped |
| replace single-scale Gaussians with full hierarchical HiSplat | models trained on RealEstate10K · RealEstate10K → Replica (zero-shot) | PSNR: +3.19 PSNR | Helped |
| add Error Aware Module (ablation) | hierarchical Gaussians on RealEstate10K · RealEstate10K | PSNR: 26.18 -> 26.76 | Helped |
| add Modulating Fusion Module (ablation) | hierarchical Gaussians with EAM on RealEstate10K · RealEstate10K | PSNR: 26.76 -> 27.02 | Helped |
| add DINOv2 features (ablation) | hierarchical Gaussians with EAM and MFM on RealEstate10K · RealEstate10K | PSNR: 27.02 -> 27.21 | Helped |
| use only stage 1 instead of all three stages (ablation) | HiSplat · RealEstate10K | PSNR: 27.21 -> 26.13 | Hurt |
| use only stages 1 and 2 instead of all three stages (ablation) | HiSplat · RealEstate10K | PSNR: 27.21 -> 26.99 | Hurt |
| use vanilla hierarchical Gaussians without inter-scale EAM and MFM (ablation) | generalizable 3D Gaussian Splatting · RealEstate10K | PSNR: -0.21 PSNR compared with MVSpalt | Hurt |
Surprising findings
- A vanilla hierarchical Gaussian representation without the error-aware and fusion modules underperforms MVSplat (26.18 vs 26.39 PSNR).
- The ablation rows show the DINOv2 feature alone gives only a small gain on RealEstate10K (27.02 to 27.21 PSNR).
Limitations
- Full three-stage model has higher inference cost and memory than MVSplat (0.51 s, 23.34 GB vs 0.27 s, 14.08 GB).
- Gains over the strongest baselines on RealEstate10K and ACID are modest (about +0.5 PSNR).
Experimental setup
Datasets: RealEstate10K ACID DTU Replica
Benchmarks: RealEstate10K ACID DTU Replica
Metrics: PSNR SSIM LPIPS
Baselines: pixelNeRF GPNR AttnRend MuRF PixelSplat MVSplat TranSplat
Builds on: pixelSplat MVSplat TranSplat MVSFormer++ DINOv2
Closest papers by meaning
- TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers (AAAI 2025 · similarity 0.92)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.91)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.91)
- GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.91)
- S2Gaussian: Sparse-View Super-Resolution 3D Gaussian Splatting (CVPR 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.