Research map / 3D vision / Gaussian splatting

TranSplat: Generalizable 3D Gaussian Splatting from Sparse Multi-View Images with Transformers

AAAI 2025 · 3D vision › Gaussian splatting · approach: Regularization and primitive design for 3DGS · 28 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

TranSplat improves feed-forward generalizable 3D Gaussian Splatting from sparse views by combining a depth-aware deformable matching transformer with monocular depth priors (Depth Anything V2) refined by a Depth Refine U-Net.

Problem

Generalizable 3D Gaussian Splatting from sparse views relies on accurate multi-view feature matching, which fails in low-texture, repetitive-pattern, and non-overlapping regions.

Approach

Uses a coarse plane-sweep depth distribution to weight depth candidates via a depth-aware deformable matching transformer, then refines depth with a U-Net conditioned on monocular relative depth from Depth Anything V2, and unprojects per-pixel depth to Gaussian centers trained with only RGB supervision (MSE + LPIPS).

Key results

• On RealEstate10K, TranSplat (300k) reports PSNR 26.69, SSIM 0.875, LPIPS 0.125, vs MVSplat 26.39 / 0.869 / 0.128.
• On ACID, TranSplat (300k) reports PSNR 28.35 vs MVSplat 28.25 and pixelSplat 28.14.
• Zero-shot Re10K→DTU: PSNR 14.93 vs MVSplat 13.94 (300k), with 2228 MB memory and 0.087 s inference.

Measured effects reported in the paper

Surprising findings

• Changing the DepthAnythingV2 size (small or large) instead of base degrades performance, which the authors attribute to balance needs between matching and depth priors.
• On ACID the gains over MVSplat are marginal (PSNR 28.35 vs 28.25), and the 200k model is slightly below MVSplat on ACID PSNR (28.20).

Limitations

• Per-pixel Gaussian prediction yields fewer points on object sides, causing gaps when rotating around objects.
• Cross-dataset gains on ACID are nearly identical to baselines, as ACID is reduced to image interpolation.
• Performance is sensitive to the monocular depth prior size.

Experimental setup

Datasets: RealEstate10K ACID

Benchmarks: RealEstate10K ACID DTU

Metrics: PSNR SSIM LPIPS memory inference time

Baselines: pixelSplat MVSplat

Builds on: MVSplat pixelSplat Depth Anything V2 Swin Transformer plane-sweep stereo

Closest papers by meaning

• DepthSplat: Connecting Gaussian Splatting and Depth (CVPR 2025 · similarity 0.92)
• HiSplat: Hierarchical 3D Gaussian Splatting for Generalizable Sparse-View Reconstruction (ICLR 2025 · similarity 0.92)
• Sparse2DGS: Geometry-Prioritized Gaussian Splatting for Surface Reconstruction from Sparse Views (CVPR 2025 · similarity 0.91)
• PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.90)
• FatesGS: Fast and Accurate Sparse-View Surface Reconstruction Using Gaussian Splatting with Depth-Feature Consistency (AAAI 2025 · similarity 0.90)
• MonoSplat: Generalizable 3D Gaussian Splatting from Monocular Depth Foundation Models (CVPR 2025 · similarity 0.90)