Research map / 3D vision / Gaussian splatting
3D Gaussian Flats: Hybrid 2D/3D Photometric Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3D Gaussian Flats, a hybrid representation that jointly optimizes constrained 2D planar Gaussians on detected flat surfaces with free-form 3D Gaussians for the rest of an indoor scene, improving depth and mesh extraction while keeping NVS quality.
Problem
Radiance-field novel view synthesis methods model flat, textureless indoor surfaces with low-density semi-transparent Gaussians, yielding poor geometry, while surface methods sacrifice visual quality.
Approach
Jointly optimize 2D Gaussians snapped to semantically-masked planes (with plane parameters optimized in alternating phases) and 3D Gaussians for freeform content, detecting planes during training via RANSAC on Gaussians and stochastically relocating nearby 3D Gaussians onto planes within an MCMC-style densification.
Key results
- On ScanNet++ iPhone-based ablation, the full model reaches 26.83 PSNR and 0.09 AbsRel, versus 3DGS-MCMC-level PSNR around 23-24 in the appendix table for comparable settings, with depth substantially improved.
- Ablation: removing plane optimization drops PSNR to 21.08 and AbsRel to 0.24; simultaneous joint optimization drops PSNR to 19.52; without relocation AbsRel rises to 0.28.
- Random point-cloud initialization achieves AbsRel 0.13 versus 0.12 with SfM initialization on DSLR scenes, with 21% vs 31% planar primitives.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the TV loss (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.34 -> 0.25 | Helped |
| add the mask loss (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.62 -> 0.25 | Helped |
| optimize plane parameters (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.54 -> 0.25 | Helped |
| alternate plane and Gaussian optimization instead of simultaneous joint optimization (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.40 -> 0.25 | Helped |
| snap Gaussians to planes (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.38 -> 0.25 | Helped |
| replace planar-scale regularization with snapping (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.36 -> 0.25 | Hurt |
| relocate Gaussians to planes (ablation) | 3D Gaussian Flats · ScanNet++ design-choice ablation | RMSE: 0.59 -> 0.25 | Helped |
| use random rather than SfM point-cloud initialization (ablation) | 3D Gaussian Flats · ScanNet++ DSLR initialization ablation | RMSE: 0.20 -> 0.21 | No clear effect |
Surprising findings
- The slight PSNR gap to unconstrained 3D methods is attributed to unconstrained geometry inflating PSNR by fitting view-dependent effects with incorrect geometry.
Limitations
- Relies on initial 3DGS reconstruction, leading to insufficient Gaussians in textureless flat areas.
- Weak spherical harmonics appearance model causes extra geometry to compensate for view-dependent effects.
- Depends on 2D semantic masks from SAM 2 and PlaneRecNet that may contain errors; RANSAC adds training overhead.
Experimental setup
Datasets: ScanNet++ ScanNetv2 ETH3D
Benchmarks: ScanNet++ ScanNetv2 ETH3D
Metrics: PSNR SSIM LPIPS RMSE MAE AbsRel depth accuracy thresholds accuracy precision recall
Baselines: 3DGS 3DGS-MCMC 2DGS PGSR RaDe-GS AirPlanes PlanarRecon Gaussian Opacity Fields DNSplatter
Builds on: 3D Gaussian Splatting 3DGS-MCMC 2D Gaussian Splatting PlaneRecNet SAM 2
Closest papers by meaning
- MAtCha Gaussians: Atlas of Charts for High-Quality Geometry and Photorealism From Sparse Views (CVPR 2025 · similarity 0.88)
- DNGaussian: Optimizing Sparse-View 3D Gaussian Radiance Fields with Global-Local Depth Normalization (CVPR 2024 · similarity 0.88)
- FatesGS: Fast and Accurate Sparse-View Surface Reconstruction Using Gaussian Splatting with Depth-Feature Consistency (AAAI 2025 · similarity 0.87)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.87)
- GaussianUDF: Inferring Unsigned Distance Functions through 3D Gaussian Splatting (CVPR 2025 · similarity 0.87)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.