Research map / 3D vision / Gaussian splatting
SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SelfSplat, a pose-free, 3D-prior-free generalizable 3D Gaussian Splatting model that jointly learns depth, relative poses, and Gaussian attributes from unposed multi-view images via self-supervised reprojection and rendering losses.
Problem
Generalizable 3D reconstruction and novel view synthesis from unposed multi-view images without pretrained 3D priors or per-scene finetuning, where existing pose-free methods rely on pretrained flow/3D models, finetuning, or expensive volume rendering.
Approach
Unifies self-supervised depth and pose estimation (reprojection loss) with pixel-aligned 3D Gaussian splatting, adding a matching-aware pose network (cross-view U-Net features with target view) and a pose-aware depth refinement module using Plücker ray embeddings, with a CroCo-v2 self-supervised monocular encoder instead of a depth prior.
Key results
- On RE10k, SelfSplat reaches 24.22 PSNR / 0.813 SSIM / 0.188 LPIPS on average, versus 21.03 PSNR for CoPoNeRF.
- On ACID, SelfSplat reaches 26.71 PSNR / 0.801 SSIM / 0.196 LPIPS, versus 25.59 PSNR for FlowCAM.
- Cross-dataset (RE10k to ACID) PSNR of 26.60 vs 23.56 for CoPoNeRF; ablation removing the reprojection loss drops PSNR by about 1.5 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the matching-aware pose network instead of removing the matching network (ablation) | SelfSplat · RE10k | PSNR: 22.21 -> 22.65 | Helped |
| use the matching-aware pose network instead of removing the matching network (ablation) | SelfSplat · RE10k | Trans. Avg.°: 15.171 -> 13.705 | Helped |
| add the depth refinement module (ablation) | SelfSplat · RE10k | PSNR: 22.05 -> 22.65 | Helped |
| add the depth refinement module (ablation) | SelfSplat · RE10k | Trans. Avg.°: 14.8 -> 13.705 | Helped |
| add reprojection loss (ablation) | SelfSplat · RE10k | PSNR: 21.12 -> 22.65 | Helped |
| add reprojection loss (ablation) | SelfSplat · RE10k | Trans. Avg.°: 28.584 -> 13.705 | Helped |
| add rendering loss (ablation) | SelfSplat · RE10k | Rotation Avg.°: 8.581 -> 1.036 | Helped |
| add rendering loss (ablation) | SelfSplat · RE10k | Trans. Avg.°: 64.436 -> 13.705 | Helped |
| use SelfSplat rather than FlowCAM | pose-free novel view synthesis · RE10k | PSNR: 22.29 -> 24.22 | Helped |
| use SelfSplat rather than FlowCAM | pose-free novel view synthesis · ACID | LPIPS: 0.294 -> 0.196 | Helped |
Surprising findings
- Removing the rendering loss causes pose translation error to rise from 13.7 to 64.4 degrees on the ablation split, showing rendering loss is critical for pose learning.
- On RE10k pose estimation, SelfSplat's average rotation error is higher than CoPoNeRF's on small-overlap pairs (1.523 vs 1.281 degrees).
Limitations
- Relies on known camera intrinsics provided from sensor metadata.
- Pose estimation accuracy on small-overlap pairs is worse than CoPoNeRF for rotation on RE10k.
Experimental setup
Datasets: RealEstate10K ACID DL3DV
Benchmarks: RealEstate10K ACID DL3DV
Metrics: PSNR SSIM LPIPS rotation error (avg/median) translation error (avg/median)
Baselines: VAE DBARF FlowCAM CoPoNeRF
Builds on: pixelSplat MVSplat CroCo self-supervised monocular depth estimation (Monodepth2) FlowCAM DBARF
Closest papers by meaning
- No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images (ICLR 2025 · similarity 0.93)
- MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.91)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.91)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.91)
- TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers (AAAI 2025 · similarity 0.90)
- KeyGS: A Keyframe-Centric Gaussian Splatting Method for Monocular Image Sequences (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.