Research map / Point clouds / Point cloud classification
LeanGaussian: Breaking Pixel or Point Cloud Correspondence in Modeling 3D Gaussians
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LeanGaussian, a DETR-style deformable Transformer that regresses 3D Gaussians as queries (with 3D centers projected as 2D reference points) for single-view novel view synthesis, avoiding pixel- or point-cloud alignment.
Problem
Single-view object-level novel view synthesis with 3D Gaussian splatting; existing feed-forward methods tie each Gaussian to a pixel or point-cloud/triplane element, wasting Gaussians and hurting geometry and texture.
Approach
Treat each decoder query as one 3D Gaussian, refine Gaussians layer by layer with deformable cross-attention, and use projected 3D Gaussian centers as 2D reference points on image features, so the number of Gaussians is decoupled from image resolution.
Key results
- On Objaverse-trained, GSO-tested single-view NVS, achieves PSNR 22.36 vs 21.06 for Splatter Image (SSIM 0.884, LPIPS 0.105).
- On ShapeNet SRN, average PSNR 25.44 (cars 25.00, chairs 25.87), outperforming Hierarchical Splatter Image (25.43 chairs, 24.18 cars).
- 3D reconstruction takes 0.14 s forward pass on a single A100 with about 10,000 Gaussians; rendering at roughly 500 FPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LeanGaussian instead of Splatter Image | single-view novel-view synthesis · GSO | PSNR: 21.06 -> 22.36 | Helped |
| Use LeanGaussian instead of Hierarchical Splatter Image | single-view novel-view synthesis · ShapeNet SRN Cars | PSNR: 24.18 -> 25.00 | Helped |
| Use LeanGaussian instead of Hierarchical Splatter Image | single-view novel-view synthesis · ShapeNet SRN Chairs | PSNR: 25.43 -> 25.87 | Helped |
| Use LeanGaussian instead of Splatter Image | single-view novel-view synthesis · GSO, far views | PSNR: 19.7071 -> 22.0127 | Helped |
| Add 3D reference points (ablation) | LeanGaussian · ShapeNet SRN Chairs | PSNR: 23.30 -> 23.45 | Helped |
| Add multi-layer refinement (ablation) | LeanGaussian · ShapeNet SRN Chairs | PSNR: 21.11 -> 23.45 | Helped |
| Add UNet features (ablation) | LeanGaussian · ShapeNet SRN Chairs | PSNR: 22.54 -> 23.45 | Helped |
| Add supervision at the first stage (ablation) | LeanGaussian · ShapeNet SRN Chairs | PSNR: 21.99 -> 23.45 | Helped |
| Initialize with feature-extractor Gaussians instead of random queries (ablation) | LeanGaussian · ShapeNet SRN Chairs | PSNR: 23.29 vs 23.45 | No clear effect |
Surprising findings
- Without the 3D reference point, PSNR drops only from 23.45 to 23.30 on chairs, a small effect in the ablation.
- Initializing the second stage from feature-extractor Gaussians rather than random queries yields nearly identical results (23.29 vs 23.45 PSNR).
Limitations
- Struggles to generate accurate textures in the background of the input view since the input image does not cover it.
- Producing clear renderings for self-occluded parts remains challenging.
Experimental setup
Datasets: ShapeNet SRN Objaverse LVIS Google Scanned Objects
Benchmarks: ShapeNet SRN Google Scanned Objects
Metrics: PSNR SSIM LPIPS inference time (s) FPS
Baselines: Splatter Image Triplane-Gaussian OpenLRM Hierarchical Splatter Image LGM PixelNeRF VisionNeRF NeRFDiff Gamba InstantMesh
Builds on: Deformable DETR Splatter Image 3D Gaussian splatting DFA3D
Closest papers by meaning
- LightGaussian: Unbounded 3D Gaussian Compression with 15x Reduction and 200+ FPS (NeurIPS 2024 · similarity 0.90)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.89)
- CoCoGaussian: Leveraging Circle of Confusion for Gaussian Splatting from Defocused Images (CVPR 2025 · similarity 0.89)
- Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis (NeurIPS 2024 · similarity 0.89)
- VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (CVPR 2024 · similarity 0.89)
- SplatFormer: Point Transformer for Robust 3D Gaussian Splatting (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.