Research map / 3D vision / Gaussian splatting
MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MaskGaussian, which treats 3D Gaussians as probabilistic entities sampled via Gumbel-Softmax masks and uses masked-rasterization so unsampled Gaussians still receive gradients, pruning 62-75% of Gaussians with little quality loss.
Problem
3DGS uses millions of Gaussians, causing high memory use; existing pruning (hand-crafted scores or learned deterministic masks) removes Gaussians based on a snapshot, leading to suboptimal reconstructions.
Approach
Model Gaussians as probabilistically existing via sampled binary masks, and apply masks inside alpha-blending (transmittance attenuation and color accumulation) rather than to opacity/scale, so masked Gaussians still receive gradients and can adjust their existence probability as the scene evolves.
Key results
- Prunes 62.4%, 67.7%, and 75.3% of Gaussians on Mip-NeRF360, Tanks & Temples, and Deep Blending, with only 0.02 PSNR drop on the first two datasets.
- Rendering speedups of 2.05x, 2.19x, and 3.16x on the three datasets; Deep Blending PSNR (29.69) slightly exceeds 3DGS (29.53) in the MaskGaussian-α setting.
- Replacing masked-rasterization with mask-times-opacity drops Mip-NeRF360 PSNR from 27.44 to 27.05 at similar Gaussian counts (0.866M vs 1.520M).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use probabilistic mask sampling and masked-rasterization instead of Compact3DGS pruning | 3DGS · Mip-NeRF360 | PSNR: 27.32 -> 27.44 | Helped |
| use probabilistic mask sampling and masked-rasterization instead of Compact3DGS pruning | 3DGS · Tanks&Temples | PSNR: 23.61 -> 23.66 | Helped |
| use probabilistic mask sampling and masked-rasterization instead of Compact3DGS pruning | 3DGS · Deep Blending | PSNR: 29.58 -> 29.76 | Helped |
| replace Gumbel Softmax with a Straight-Through Estimator (ablation) | MaskGaussian · Mip-NeRF360 | PSNR: 27.44 -> 27.30 | Hurt |
| increase λm from 0.0005 to 0.001 with Gumbel Softmax (ablation) | MaskGaussian · Mip-NeRF360 | #GS (M): 1.520 -> 1.171 | Helped |
| replace masked-rasterization with multiplying the mask by opacity (ablation) | MaskGaussian · Mip-NeRF360 | PSNR: 27.44 -> 27.05 | Hurt |
| relax λm for mask-opacity multiplication (ablation) | MaskGaussian · Mip-NeRF360 | PSNR: 27.05 -> 27.37 | No clear effect |
| increase λm from 0.0005 to 0.001 with masked-rasterization (ablation) | MaskGaussian · Mip-NeRF360 | #GS (M): 1.520 -> 1.171 | Helped |
| apply masks throughout training with Ours-β settings (ablation) | MaskGaussian · Mip-NeRF360 | PSNR: 27.32 -> 27.44 | Helped |
| apply masks throughout training with Ours-β settings (ablation) | MaskGaussian · Deep Blending | PSNR: 29.58 -> 29.76 | Helped |
| use MaskGaussian rather than Compact3DGS with Taming-3DGS | Taming-3DGS · Garden | PSNR: 26.694 -> 26.717; 27.318 -> 27.391; 27.477 -> 27.522; 27.551 -> 27.616 | Mixed |
Surprising findings
- Multiplying masks with opacity causes a 'death spiral' where unsampled Gaussians stop receiving gradients and are eventually discarded, yielding a large quality drop.
- Pruning improves Deep Blending quality over the original 3DGS, which the authors attribute to a regularization effect.
Limitations
- Pruning requires training-time mask sampling with a balancing hyperparameter lambda_m that must be tuned per setting
- Gains over Compact3DGS are shown mainly in the paper's own reproduction; baselines use only pruning components
Experimental setup
Datasets: Mip-NeRF360 Tanks & Temples Deep Blending
Benchmarks: Mip-NeRF360 Tanks & Temples Deep Blending
Metrics: PSNR SSIM LPIPS number of Gaussians FPS training time model size peak GPU memory
Baselines: 3DGS Compact3DGS RadSplat LightGaussian Taming-3DGS Mask x opacity
Builds on: 3D Gaussian Splatting Compact3DGS Gumbel-Softmax LightGaussian RadSplat
Closest papers by meaning
- LP-3DGS: Learning to Prune 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.92)
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (NeurIPS 2024 · similarity 0.92)
- PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting (CVPR 2025 · similarity 0.92)
- Optimized Minimal 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
- VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (CVPR 2024 · similarity 0.92)
- 3D Gaussian Rendering Can Be Sparser: Efficient Rendering via Learned Fragment Pruning (NeurIPS 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.