Research map / 3D vision / NeRF and radiance fields
Omni-Recon: Harnessing Image-based Rendering for General-Purpose Neural Radiance Fields
Yonggan Fu, Huaizhi Qu, Zhifan Ye, Chaojian Li, Kevin Zhao, Yingyan (Celine) Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| reuse RGB blending weights to blend 2D predictions (predict-then-blend) instead of predicting on rendered images (ablation) | Omni-Recon · ScanNet zero-shot semantic segmentation | mIoU: 41.32 -> 61.11 | Improved |
| predict-then-blend instead of render-then-predict (ablation) | Omni-Recon · Replica zero-shot semantic segmentation | mIoU: 15.64 -> 32.11 | Improved |
| predict-then-blend instead of render-then-predict (ablation) | Omni-Recon · ScanNet keypoint 3D | l1 error: 0.0412 -> 0.0197 | Improved |
| add LoRA parameter-efficient tuning on top of zero-shot | Omni-Recon · ScanNet semantic segmentation | mIoU: 61.11 -> 62.35 | Improved |
| add LoRA PET to edge and keypoint tasks | Omni-Recon · ScanNet edge detection | l1 error: 0.0434 -> 0.0342 | Improved |
| finetune mesh and shader with differentiable rendering for 10 seconds | Omni-Recon real-time pipeline · DTU | PSNR (mean): 22.96 -> 25.68 | Improved |
| longer finetuning (1 min to 5 min) of mesh and shader | Omni-Recon real-time pipeline · DTU | PSNR (mean): 28.34 -> 29.02 | Improved |
| bake geometry branch into mesh and use appearance branch as shader (Nvdiffrast) | Omni-Recon · DTU | FPS: 0.024 (ReTR) -> 71.3 | Improved |
| transformer geometry branch with progressive fusion, versus prior generalizable reconstruction | Omni-Recon · DTU sparse-view reconstruction | mean Chamfer distance: 1.17 -> 1.13 | Improved |
| Omni-Recon versus ReTR on novel view rendering | Omni-Recon · DTU novel view rendering | PSNR (mean): 25.59 -> 26.32 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- VolRecon — VolRecon: Volume Rendering of Signed Ray Distance Functions for Generalizable Multi-View Reconstruction (CVPR 2023) · better on 3 of 3 shared results
- ReTR — ReTR: Modeling Rendering Via Transformer for Generalizable Neural Surface Reconstruction (NeurIPS 2023) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Learning Robust Generalizable Radiance Field with Visibility and Feature Augmented Point Representation (ICLR 2024 · similarity 0.91)
- GRF: Learning a General Radiance Field for 3D Representation and Rendering (ICCV 2021 · similarity 0.90)
- VD-NeRF: Visibility-Aware Decoupled Neural Radiance Fields for View-Consistent Editing and High-Frequency Relighting (TPAMI 2025 · similarity 0.90)
- Reducing Shape-Radiance Ambiguity in Radiance Fields with a Closed-Form Color Estimation Method (NeurIPS 2023 · similarity 0.90)
- Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.89)
- Mask-Based Modeling for Neural Radiance Fields (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
