Research map / 3D vision / Gaussian splatting
FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeSplat, a feed-forward generalizable 3DGS model that uses low-cost cost-volume aggregation and pixel-wise triplet fusion plus free-view training to reconstruct global 3D Gaussians from long indoor image sequences without depth supervision.
Problem
Existing generalizable 3D Gaussian Splatting methods are limited to narrow-range stereo interpolation because heavy transformer backbones cannot scale to long sequences, leading to poorly localized Gaussians, floaters at extrapolated views, and redundant Gaussians in overlapping regions.
Approach
Use a lightweight CNN backbone with adaptive pose-proximity cost volumes and multi-scale UNet++-style decoding to predict depth and Gaussian triplets, then progressively fuse local triplets into global ones via pixel-wise projection alignment and GRU feature updates, trained with a free-view strategy that samples 2-8 context views and supervises broader view ranges.
Key results
- On ScanNet 3-view interpolation, FreeSplat-spec reaches 27.45 PSNR vs. 26.68 for MVSplat, with inference time 0.121s vs. 0.192s.
- On ScanNet 10-view long sequence, FreeSplat-fv reaches 25.90 PSNR interpolation and 24.64 PSNR extrapolation, versus 22.78 and 21.60 for MVSplat.
- The PTF module reduces the number of Gaussians by about 55.0% on long sequences.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FreeSplat instead of MVSplat | ScanNet, 2 views · ScanNet, 2 views | PSNR: 27.27 -> 28.08 | Helped |
| Use FreeSplat instead of MVSplat | ScanNet, 10-view interpolation · ScanNet, 10 views, interpolation | PSNR: 22.78 -> 25.90 | Helped |
| Add Free-View Training | FreeSplat on ScanNet, 10-view extrapolation · ScanNet, 10 views, extrapolation | PSNR: 23.78 -> 24.64 | Helped |
| Add Free-View Training | FreeSplat on ScanNet, 3 views · ScanNet, 3 views | PSNR: 27.45 -> 27.34 | Hurt |
| Add Pixel-wise Triplet Fusion (ablation) | FreeSplat with cost volume, ScanNet 3 views · ScanNet, 3 views | PSNR: 27.12 -> 27.45 | Helped |
| Add Pixel-wise Triplet Fusion (ablation) | FreeSplat with cost volume, ScanNet 10 views · ScanNet, 10 views | PSNR: 24.23 -> 25.15 | Helped |
| Add Free-View Training (ablation) | FreeSplat with cost volume and PTF, ScanNet 10 views · ScanNet, 10 views | PSNR: 25.15 -> 25.90 | Helped |
| Add Free-View Training (ablation) | FreeSplat with cost volume and PTF, ScanNet 3 views · ScanNet, 3 views | PSNR: 27.45 -> 27.34 | Hurt |
| Add cost volume to PTF (ablation) | FreeSplat, ScanNet 3 views · ScanNet, 3 views | PSNR: 22.10 -> 27.45 | Helped |
| Add cost volume to PTF (ablation) | FreeSplat, ScanNet 10 views · ScanNet, 10 views | PSNR: 17.94 -> 25.15 | Helped |
| Use FreeSplat instead of MVSplat | Replica, 10 views · Replica, 10 views | PSNR: 18.66 -> 21.95 | Helped |
| Add PTF (ablation) | FreeSplat-fv on 30-view whole-scene reconstruction · 30 input views | FPS: 39 to 72 FPS | Helped |
Surprising findings
- Without PTF, the cost-volume-only variant performs much better than PTF-only, and adding FVT to the cost volume alone lowers 3-view PSNR from 27.12 to 26.41, suggesting FVT and PTF interact non-trivially.
- On RE10K and ACID with 2 views, the method is only on par with MVSplat and below it on ACID PSNR, indicating it is not designed for sparse-view inputs.
Limitations
- GPU memory exceeds 40GB for sequences longer than 50 images.
- Unsupervised depth leaves a gap versus methods using 3D supervision or RGB-D inputs, with poor performance on textureless and specular regions.
- Not designed for highly sparse-view inputs, as shown by weaker results on RE10K and ACID with 2 views.
Experimental setup
Datasets: ScanNet Replica RE10K ACID
Benchmarks: ScanNet Replica RE10K ACID
Metrics: PSNR SSIM LPIPS depth Abs Diff depth Abs Rel delta<1.25 inference time number of Gaussians FPS
Baselines: pixelSplat MVSplat NeuRay SurfelNeRF GGRt
Builds on: 3D Gaussian Splatting pixelSplat MVSplat UNet++
Closest papers by meaning
- MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.91)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.91)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.91)
- SPC-GS: Gaussian Splatting with Semantic-Prompt Consistency for Indoor Open-World Free-view Synthesis from Sparse Inputs (CVPR 2025 · similarity 0.91)
- SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.