Research map / 3D vision / Gaussian splatting
MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a feed-forward generalizable 3D Gaussian Splatting model on a frozen Depth Anything (ViT-S) encoder, adding a Mono-Multi feature adapter and integrated monocular features in the cost volume and refinement.
Problem
Generalizable feed-forward 3D Gaussian Splatting from sparse posed views struggles on unfamiliar scenes because learned multi-view features generalize poorly beyond the training distribution.
Approach
Repurpose a frozen pretrained monocular depth foundation model as the backbone: a DPT-based adapter plus a multi-view cross-attention transformer converts monocular features into cross-view features, and monocular features are injected into both the plane-sweep cost volume and the Gaussian refinement network.
Key results
- On RealEstate10K, MonoSplat reaches 26.68 PSNR / 0.875 SSIM / 0.123 LPIPS, beating MVSplat (26.39 / 0.869 / 0.128).
- Zero-shot Re10k→DTU: 15.25 PSNR / 0.605 SSIM / 0.291 LPIPS vs. MVSplat 13.94 / 0.473 / 0.385.
- Ablation: removing monocular features drops Re10k→DTU PSNR to 10.33 from 15.24; total params 30.3M with 10.3M trainable.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Keep the depth foundation model backbone frozen instead of fine-tuning it (ablation) | MonoSplat · Re10k→DTU | PSNR: 12.63 -> 15.24 | Helped |
| Add DPT multi-scale feature aggregation (ablation) | MonoSplat · Re10k | PSNR: 25.25 -> 26.50 | Helped |
| Add cross-view feature aggregation (ablation) | MonoSplat · Re10k→DTU | PSNR: 10.28 -> 15.24 | Helped |
| Add monocular feature integration in both the cost volume and feature refinement (ablation) | MonoSplat · Re10k→DTU | PSNR: 10.33 -> 15.24 | Helped |
| Add monocular feature integration in feature refinement as well as the cost volume (ablation) | MonoSplat · Re10k | PSNR: 26.20 -> 26.50 | Helped |
| Add monocular feature integration in the cost volume as well as feature refinement (ablation) | MonoSplat · Re10k | PSNR: 26.06 -> 26.50 | Helped |
Surprising findings
- Fine-tuning the depth backbone instead of freezing it degrades in-domain and especially cross-domain generalization (2.61 dB drop on DTU).
Limitations
- Total parameter count (30.3M) is higher than MVSplat (12.0M) because of the frozen depth model.
- Evaluation restricted to 256x256 resolution and to a few datasets (RealEstate10K, ACID, DTU).
Experimental setup
Datasets: RealEstate10K ACID DTU
Benchmarks: RealEstate10K ACID DTU
Metrics: PSNR SSIM LPIPS inference time parameters memory
Baselines: pixelSplat MVSplat latentSplat eFreeSplat MuRF pixelNeRF GPNR Du et al.
Builds on: pixelSplat MVSplat Depth Anything DPT Swin Transformer
Closest papers by meaning
- DepthSplat: Connecting Gaussian Splatting and Depth (CVPR 2025 · similarity 0.92)
- SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.91)
- Splatter Image: Ultra-Fast Single-View 3D Reconstruction (CVPR 2024 · similarity 0.91)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.91)
- CompactSplat: Spatially Adaptive Gaussian Distribution for Feedforward Scene Reconstruction (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.