Research map / 3D vision / Gaussian splatting
pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Feed-forward pixelSplat predicts pixel-aligned 3D Gaussians from two images, using an epipolar transformer for scale and a reparameterized probabilistic depth sampling to avoid local minima.
Problem
Generalizable wide-baseline novel view synthesis from sparse image pairs is slow and memory-heavy with light-field transformers, and primitive-based 3D representations suffer local minima and scale ambiguity in SfM-posed real-world data.
Approach
Predict pixel-aligned 3D Gaussians in a single forward pass by parameterizing Gaussian means as samples from a per-pixel discrete depth distribution, with opacity set to the sampled bucket probability (reparameterization so gradients flow), and resolve per-scene scale via an epipolar transformer that attends with positionally encoded depths.
Key results
- On RealEstate10k, pixelSplat reaches 26.09 PSNR / 0.863 SSIM / 0.136 LPIPS, versus 24.78 PSNR for Du et al.
- On ACID, pixelSplat reaches 28.27 PSNR / 0.843 SSIM / 0.146 LPIPS, outperforming all baselines.
- Rendering is about 650x faster than the next-fastest baseline (Du et al.) when encoding a scene and decoding 100 frames, and rendering takes 0.002 s per frame.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use pixelSplat instead of Du et al. [10] | novel view synthesis · ACID | PSNR: 26.88 -> 28.27 | Helped |
| use pixelSplat instead of Du et al. [10] | novel view synthesis · RealEstate10k | PSNR: 24.78 -> 26.09 | Helped |
| remove the epipolar encoder (ablation) | pixelSplat · RealEstate10k | PSNR: 26.09 -> 19.89 | Hurt |
| remove depth positional encoding from epipolar attention (ablation) | pixelSplat · RealEstate10k | PSNR: 26.09 -> 24.97 | Hurt |
| replace probabilistic Gaussian depth sampling with direct depth regression (ablation) | pixelSplat · RealEstate10k | PSNR: 26.09 -> 24.62 | Hurt |
| add depth regularization (ablation) | pixelSplat · RealEstate10k | PSNR: 26.09 -> 25.28 | Hurt |
Surprising findings
- Removing the epipolar encoder drops RealEstate10k PSNR from 26.09 to 19.89, showing scale ambiguity dominates performance.
Limitations
- Reflective surfaces appear transparent and Gaussians look billboard-like from out-of-distribution views.
- Requires camera poses (SfM-computed, scale-ambiguous) as input.
Experimental setup
Datasets: RealEstate10k ACID
Benchmarks: RealEstate10k ACID
Metrics: PSNR SSIM LPIPS inference time memory
Baselines: pixelNeRF GPNR Du et al. light field transformer
Builds on: 3D Gaussian splatting epipolar transformer pixelNeRF variational autoencoder reparameterization
Closest papers by meaning
- 3D Convex Splatting: Radiance Field Rendering with 3D Smooth Convexes (CVPR 2025 · similarity 0.92)
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.92)
- COLMAP-Free 3D Gaussian Splatting (CVPR 2024 · similarity 0.92)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.91)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.91)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.