Research map / 3D vision / Gaussian splatting
PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PF3plat, a feed-forward pose-free 3DGS framework that coarsely aligns pixel-aligned Gaussians using pretrained monocular depth and correspondence networks, then refines depth and pose with lightweight learnable modules and geometry-aware confidence.
Problem
Feed-forward novel view synthesis from sparse, unposed images with minimal overlap, where misaligned pixel-aligned 3D Gaussian centers cause noisy or sparse gradients that destabilize training.
Approach
Coarse alignment of pixel-aligned Gaussians via off-the-shelf monocular depth (UniDepth) and correspondence (LoFTR-style) models plus robust pose solving, followed by learnable depth offset and pose refinement modules and a geometry-aware confidence score (from multi-view and guidance cost volumes) that conditions Gaussian parameter prediction; training uses photometric, 2D-3D and 3D-3D consistenc
Key results
- On RealEstate10K, PF3plat reaches 23.589 average PSNR, about 4.0 dB above the pose-free CoPoNeRF (19.536).
- On ACID, PF3plat reaches 25.640 average PSNR versus 22.440 for CoPoNeRF.
- With 2 views, inference takes about 0.39 s versus 17.29 s for CoPoNeRF in the speed table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PF3plat instead of CoPoNeRF | Pose-free novel view synthesis · RealEstate10K Avg | PSNR: 19.536 -> 23.589 | Helped |
| Use PF3plat instead of CoPoNeRF | Pose-free novel view synthesis · ACID Avg | PSNR: 22.440 -> 25.640 | Helped |
| Use PF3plat instead of CoPoNeRF | Pose-free novel view synthesis · DL3DV Small | PSNR: 15.509 -> 19.822 | Helped |
| Use PF3plat instead of CoPoNeRF | Pose-free novel view synthesis · DL3DV Large | PSNR: 17.586 -> 22.668 | Helped |
| Add depth refinement (ablation) | PF3plat · RealEstate10K | PSNR: 22.012 -> 23.589 | Helped |
| Add pose refinement (ablation) | PF3plat · RealEstate10K | PSNR: 21.623 -> 23.589 | Helped |
| Add geometry confidence (ablation) | PF3plat · RealEstate10K | PSNR: 21.443 -> 23.589 | Helped |
| Add monocular depth network (ablation) | PF3plat · RealEstate10K | PSNR: 16.132 -> 23.589 | Helped |
| Add triangulation consistency loss (ablation) | PF3plat · RealEstate10K | PSNR: 19.001 -> 23.589 | Helped |
| Add regularization loss (ablation) | PF3plat · RealEstate10K | PSNR: 21.332 -> 23.589 | Helped |
| Remove correspondence network (ablation) | PF3plat · RealEstate10K | PSNR | Hurt |
| Fully fine-tune the depth network (ablation) | PF3plat · RealEstate10K | PSNR | Hurt |
Surprising findings
- Trails MASt3R in pose estimation on ACID, attributed to large-scale and dynamic scenes.
- Full fine-tuning of the depth network fails, and removing the monocular depth network drops PSNR from 23.589 to 16.132.
Limitations
- Pose estimation lags MASt3R on ACID.
- Relies on frozen pretrained depth and correspondence networks; end-to-end depth fine-tuning causes failure.
- Pairwise robust pose solving makes 12-view inference slower than DBARF.
Experimental setup
Datasets: RealEstate10K ACID DL3DV
Benchmarks: RealEstate10K ACID DL3DV
Metrics: PSNR SSIM LPIPS rotation error translation error ATE
Baselines: DBARF FlowCAM CoPoNeRF PixelSplat MVSplat InstantSplat CF-3DGS MASt3R
Builds on: MVSplat pixelSplat CoPoNeRF UniDepth DBARF
Closest papers by meaning
- No Pose, No Problem: Surprisingly Simple 3D Gaussian Splats from Sparse Unposed Images (ICLR 2025 · similarity 0.92)
- pixelSplat: 3D Gaussian Splats from Image Pairs for Scalable Generalizable 3D Reconstruction (CVPR 2024 · similarity 0.91)
- FreeSplat: Generalizable 3D Gaussian Splatting Towards Free View Synthesis of Indoor Scenes (NeurIPS 2024 · similarity 0.91)
- SelfSplat: Pose-Free and 3D Prior-Free Generalizable 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.90)
- TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers (AAAI 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.