Research map / 3D vision / Gaussian splatting
UniSHARP: Universal Sharp Monocular View Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends the feedforward monocular 3D Gaussian regressor SHARP to universal cameras (perspective, wide-FoV, fisheye, panoramic) via a ray-distance Gaussian representation, a UniK3D-based encoder, and a new FoV-stratified benchmark with OmniRooms.
Problem
Monocular novel view synthesis from a single image fails to generalize beyond narrow perspective cameras because pinhole-trained feedforward Gaussian regressors like SHARP assume pinhole geometry, so wide-FoV, fisheye and panoramic inputs are poorly handled.
Approach
Predict Gaussians in a unified ray-distance space (unit rays plus radial distance) with two radial layers, anchored to native ray grids and refined by residuals decoded from fused 2D semantic and 3D UniK3D features, trained under mixed-camera supervision with latitude-dependent dropout for panoramas.
Key results
- On WildRGB-D, UniSHARP reaches 21.556 PSNR / 0.674 SSIM / 0.143 LPIPS, versus 18.300 PSNR for SHARP.
- On panoramic HM3D, UniSHARP reaches 29.244 PSNR / 0.895 SSIM / 0.065 LPIPS, versus 23.398 PSNR for the best baseline Matrix3D.
- UniSHARP runs in 3.1 s per sample versus 8.6 s for PanoDreamer and 38.8 s for Matrix3D.
Surprising findings
- Removing the depth-RGB-to-feature input (replacing learned features with direct depth-RGB inputs) causes the largest degradation among model-design ablations.
- Pose-free variant is only modestly worse than calibrated model (20.85 vs 21.56 PSNR on WildRGB-D).
Limitations
- Limited ability to hallucinate content completely outside the source image; holes may appear near outer boundaries for large target motion.
- Evaluation restricted to local motion (overlap >60%, camera distance <0.5 m, index gap <10), so long-range extrapolation is not measured.
- Pseudo depth labels from UniK3D used for datasets lacking ground-truth depth.
Experimental setup
Datasets: RealEstate10K DL3DV WildRGB-D OmniRooms-Wide ScanNet++ Fisheye HM3D OmniRooms Tanks and Temples Replica
Benchmarks: WildRGB-D DL3DV RealEstate10K Tanks and Temples HM3D OmniRooms Replica OmniRooms-Wide ScanNet++ Fisheye
Metrics: PSNR SSIM LPIPS runtime
Baselines: SHARP Flash3D LVSM TMPI PanoDreamer Matrix3D
Builds on: SHARP UniK3D 3D Gaussian Splatting 3DGEER
Closest papers by meaning
- UniK3D: Universal Camera Monocular 3D Estimation (CVPR 2025 · similarity 0.88)
- UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.85)
- Uni3D: Exploring Unified 3D Representation at Scale (ICLR 2024 · similarity 0.85)
- UniDepth: Universal Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.85)
- UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Rendering (NeurIPS 2026 · similarity 0.85)
- MultiDiff: Consistent Novel View Synthesis from a Single Image (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.