Research map / Point clouds / Point cloud classification
GaussianPainter: Painting Point Cloud into 3D Gaussians with Normal Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GaussianPainter, a feed-forward network that paints a point cloud into anisotropic 3D Gaussians conditioned on a reference image, using predicted surface normals to fix Gaussian rotations and a multiscale triplane for appearance injection.
Problem
Feed-forward prediction of 3D Gaussian parameters from point clouds is unstable because of the non-uniqueness of the large Gaussian parameter space (rotation, anisotropic scale, SH), while existing Gaussian generators rely on slow test-time optimization or denoising.
Approach
Predict a surface normal per point (via isotropic normal rendering supervised on 2D normal maps) and derive each Gaussian's rotation from it, which constrains the parameter space so the network can predict the remaining anisotropic scales, opacity and colors in one forward pass; appearance from a reference image is injected via a multiscale DINOv2-based triplane.
Key results
- On OmniObject3D novel view synthesis, GaussianPainter reaches 30.9 PSNR / 0.945 SSIM / 0.134 LPIPS versus 28.4 / 0.914 / 0.177 for AGG and 25.8 / 0.881 / 0.191 for TriplaneGaussian.
- Ablation: normal-guided rotation gives 30.9 PSNR vs 28.4 for isotropic (no rotation) and 25.8 for rotation predicted from scratch.
- Performance degrades gracefully with point cloud noise: 27.5 PSNR at 0% noise, 23.7 at 90% noisy points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use GaussianPainter instead of AGG | OmniObject3D novel view synthesis · OmniObject3D valid split | PSNR: 28.4 -> 30.9 | Helped |
| use GaussianPainter instead of TriplaneGaussian | OmniObject3D novel view synthesis · OmniObject3D valid split | LPIPS: 0.191 -> 0.134 | Helped |
| replace direct rotation prediction from scratch with normal-guided rotation (ablation) | GaussianPainter · OmniObject3D validation split | PSNR: 25.8 -> 30.9 | Helped |
| initialize rotation prediction from a normal-guided model and fine-tune (ablation) | GaussianPainter · OmniObject3D validation split | PSNR: 25.8 -> 29.5 | Helped |
| add rotation rather than use isotropic scale (ablation) | GaussianPainter · OmniObject3D validation split | PSNR: 28.4 -> 30.9 | Helped |
| add multi-scale triplane rather than single-scale triplane (ablation) | GaussianPainter without cross-modal attention · OmniObject3D | PSNR: 29.2 -> 30.6 | Helped |
| add cross-modal attention to multi-scale triplane (ablation) | GaussianPainter · OmniObject3D | PSNR: 30.6 -> 30.9 | Helped |
| use multi-scale triplanes and cross-modal attention rather than attention-only design (ablation) | GaussianPainter · OmniObject3D | PSNR: 27.4 -> 30.9 | Helped |
| increase point cloud noise from 0% to 90% | GaussianPainter · Noisy point clouds | PSNR: 27.5 -> 23.7 | Hurt |
| use normal-guided rotation rather than unconstrained rotation (ablation) | GaussianPainter rotation representation · Multi-view image optimization | PSNR: 29.16 -> 28.70 | Hurt |
| add anisotropic rotation rather than use isotropic scale (ablation) | GaussianPainter rotation representation · Multi-view image optimization | PSNR: 26.30 -> 28.70 | Helped |
Surprising findings
- Using features from shallower DINOv2 layers at inference (not used in training) still yields reasonable and sometimes better painting results, providing diversity.
Limitations
- Not applied to large-scale scene-level point clouds.
- Baselines were re-implemented with ground-truth point clouds; AGG is not open-source, so comparison is approximate.
- Performance drops notably under heavy point cloud noise (23.7 PSNR at 90% noisy points).
Experimental setup
Datasets: OmniObject3D Objaverse
Benchmarks: OmniObject3D validation split
Metrics: PSNR SSIM LPIPS
Baselines: TriplaneGaussian AGG
Builds on: 3D Gaussian splatting DINOv2 Triplane Gaussian AGG
Closest papers by meaning
- Sparse Point Cloud Patches Rendering via Splitting 2D Gaussians (CVPR 2025 · similarity 0.90)
- SuGaR: Surface-Aligned Gaussian Splatting for Efficient 3D Mesh Reconstruction and High-Quality Mesh Rendering (CVPR 2024 · similarity 0.90)
- 3D Gaussian Splatting as Markov Chain Monte Carlo (NeurIPS 2024 · similarity 0.90)
- Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.90)
- Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.89)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.