Research map / 3D vision / 3D generation
UFORecon: Generalizable Sparse-View Surface Reconstruction from Arbitrary and Unfavorable Sets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UFORecon, a generalizable sparse-view neural implicit surface reconstruction method that uses cross-view matching transformers, cascaded correlation frustums, and explicit similarity encoding plus random-set training to stay robust under arbitrary and unfavorable view combinations.
Problem
Generalizable neural implicit surface reconstruction from sparse views degrades sharply when input view combinations are unfavorable (low overlap, large baselines), because existing methods train and test only on best-selected view combinations.
Approach
Combine cross-view matching transformer features with cascaded correlation frustums (MVS-style 3D volumes) and explicit pairwise cosine-similarity encodings as view-consistent priors, and train with random source-view sets instead of only favorable combinations.
Key results
- On DTU unfavorable 3-view sets, UFORecon achieves mean Chamfer Distance 1.56 versus 3.18 for VolRecon and 2.94 for ReTR; with random set training (Ours*) it reaches 1.28.
- On DTU favorable sets, UFORecon reaches mean CD 0.99 versus 1.38 for VolRecon and 1.17 for ReTR, showing gains even under the conventional protocol.
- Adding explicit similarity encoding to VolRecon via MatchNeRF lowers unfavorable CD from 3.18 to 2.47, while UFORecon attains 1.56 with only 1.5M parameters.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use random set training instead of best-set training | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 1.56 -> 1.28 | Helped |
| use random set training instead of best-set training | UFORecon · DTU favorable sets | Chamfer Distance (CD): 0.99 -> 1.01 | Hurt |
| add cross-view correlation frustum (ablation) | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 3.19 -> 1.56 | Helped |
| use correlation frustum L=1 instead of the full configuration (ablation) | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 1.56 -> 1.91 | Hurt |
| use correlation frustum L=2 instead of the full configuration (ablation) | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 1.56 -> 1.71 | Hurt |
| add similarity encoding (ablation) | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 1.62 -> 1.56 | Helped |
| add depth supervision (ablation) | UFORecon · DTU unfavorable sets | Chamfer Distance (CD): 2.26 -> 1.56 | Helped |
| add MatchNeRF matching features to VolRecon | VolRecon · DTU favorable sets | Chamfer Distance (CD): 1.38 -> 1.26 | Helped |
| add MatchNeRF matching features to VolRecon | VolRecon · DTU unfavorable sets | Chamfer Distance (CD): 3.18 -> 2.47 | Helped |
Surprising findings
- Directly applying random set training to VolRecon destabilizes training and yields suboptimal results, whereas it helps UFORecon.
Limitations
- Struggles to reconstruct scenes of much higher complexity.
- Assumes ground-truth camera poses are given.
Experimental setup
Datasets: DTU
Benchmarks: DTU BlendedMVS
Metrics: Chamfer Distance
Baselines: SparseNeuS VolRecon ReTR C2F2NeuS PixelNeRF IBRNet MVSNeRF VolSDF NeuS COLMAP TransMVSNet
Builds on: NeuS VolRecon C2F2NeuS MatchNeRF TransMVSNet GMFlow
Closest papers by meaning
- NeuSurf: On-Surface Priors for Neural Surface Reconstruction from Sparse Input Views (AAAI 2024 · similarity 0.87)
- GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction (AAAI 2025 · similarity 0.87)
- Differentiable Neural Surface Refinement for Modeling Transparent Objects (CVPR 2024 · similarity 0.86)
- PMNI: Pose-free Multi-view Normal Integration for Reflective and Textureless Surface Reconstruction (CVPR 2025 · similarity 0.86)
- NeRSP: Neural 3D Reconstruction for Reflective Objects with Sparse Polarized Images (CVPR 2024 · similarity 0.86)
- 3D Reconstruction with Generalizable Neural Fields using Scene Priors (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.