Research map / 3D vision / Gaussian splatting

UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Rendering

NeurIPS 2026 · 3D vision › Gaussian splatting · approach: Compact and efficient Gaussian splatting · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes UniGS, a Scaffold-GS-based 3D Gaussian Splatting framework that jointly renders RGB, depth (via ray-ellipsoid midpoint intersection), normals (finite differences on depth), and semantic logits, plus a learnable gradient-factor attribute for differentiable pruning.

Problem

Existing 3DGS methods render RGB well but use Gaussian centers for depth, giving poor geometric consistency and surface normals, produce redundant Gaussians, and handle multimodal outputs (depth, normal, semantics) in separate pipelines.

Approach

Render depth from the midpoint of ray-ellipsoid intersections so rotation and scale receive analytic depth gradients; derive normals by finite differences of backprojected depth; add a per-Gaussian gradient factor k driven toward 1 whose deviation drives differentiable pruning; render all modalities in one rasterization pass.

Key results

• On Replica O1S2 sequence, depth Abs.Rel of 0.0015 vs 0.0154 for Normal-GS (the strongest baseline in that table).
• On Replica-Room0 SGS-SLAM comparison, training time 32.2 min vs 212.6 min, with PSNR 36.66 vs 36.28 and mIoU 0.987 vs 0.976.
• Gaussian count on Mip-NeRF360 reduced to 161081 vs 318064 for Normal-GS, with FPS 169.8 vs 180.3 (slightly lower).

Surprising findings

• Reported FPS is somewhat lower than Normal-GS and Scaffold-GS on several datasets despite the pruning, which the authors attribute to multimodal rendering cost.
• Depth supervision for Mip-NeRF360, Tanks & Temples and Deep Blending comes from Normal-GS outputs rather than true geometry, so depth/normal numbers there are relative to a pseudo-ground-truth.

Limitations

• Geometry supervision on real datasets is taken from Normal-GS outputs and semantic labels from OpenSeed, so no true geometric or semantic ground truth is used there.
• Frame rate drops relative to some baselines because all modalities are rendered simultaneously.
• Normal computation is a post-hoc finite-difference step on depth rather than a direct rasterized normal, and the paper's own RGB results are mixed versus baselines.

Experimental setup

Datasets: Mip-NeRF 360 Tanks & Temples Deep Blending Replica

Benchmarks: Replica Mip-NeRF 360 Tanks & Temples Deep Blending

Metrics: PSNR SSIM LPIPS Abs.Rel RMSE cosine similarity mIoU Gaussian count FPS

Baselines: 3DGS Scaffold-GS 2DGS Normal-GS GeoGaussian SGS-SLAM Mip-NeRF iNGP

Builds on: 3D Gaussian Splatting Scaffold-GS Normal-GS SGS-SLAM

Closest papers by meaning

• Evolving High-Quality Rendering and Reconstruction in a Unified Framework with Contribution-Adaptive Regularization (CVPR 2025 · similarity 0.90)
• Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.90)
• Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS (NeurIPS 2025 · similarity 0.90)
• 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes (CVPR 2025 · similarity 0.90)
• Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering (NeurIPS 2024 · similarity 0.90)
• pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.90)