Research map / 3D vision / NeRF and radiance fields
ICON: Incremental CONfidence for Joint Pose and Radiance Field Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ICON, an incremental, confidence-weighted joint optimization of NeRF and camera poses from monocular RGB video that removes the need for SfM pose initialization.
Problem
Training NeRFs requires accurate camera poses (usually from SfM), and existing joint pose-and-NeRF methods fail when initial poses are unavailable and camera trajectories are large, as in 360-degree object flyarounds.
Approach
Register video frames incrementally with a smooth-motion prior and learn a Neural Confidence Field on top of NeRF; pose confidence weights NeRF gradients and ray confidence weights pose gradients, with confidence-triggered restarts and a confidence-weighted Sampson epipolar constraint.
Key results
- On CO3D full-scene, ICON reaches ATE 0.138 and ATErot 1.16 with PSNR 22.24, versus BARF ATE 6.215 and ATErot 114.63 and PSNR 12.77.
- On CO3D object-only (no background), ICON achieves ATErot 1.80 and PSNR 22.45, outperforming COLMAP-SPSG and DROID-SLAM pose baselines.
- On HO3D, ICON (RGB only) achieves ATE 0.033 versus BundleSDF (RGB-D) 0.021; mesh Chamfer distance 0.7 cm vs BundleSDF 0.77 cm.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use incremental registration and confidence-guided optimization instead of BARF | Joint pose and NeRF optimization · CO3D full-scene | ATE: 6.215 -> 0.138 | Helped |
| Use incremental registration and confidence-guided optimization instead of BARF | Joint pose and NeRF optimization · CO3D object-only | ATE: 6.522 -> 0.215 | Helped |
| Use RGB-only ICON instead of RGB-D BundleSDF | HO3D pose tracking · HO3D | ATE: 0.021 -> 0.033 | Hurt |
| Use ICON instead of BARF | Novel view synthesis · LLFF | PSNR: 23.89 -> 24.23 | Helped |
| Remove confidence-based restart (ablation) | ICON · CO3D full-scene | ATE: 0.138 -> 0.714 | Hurt |
| Remove a reported ICON component (ablation) | ICON · CO3D full-scene | ATE: 0.138 -> 1.691 | Hurt |
| Remove reported ICON components (ablation) | ICON · CO3D full-scene | ATE: 0.138 -> 1.283 | Hurt |
| Remove reported ICON components (ablation) | ICON · CO3D full-scene | ATE: 0.138 -> 3.075 | Hurt |
| Remove incremental registration, geometric constraint, confidence calibration, and restarts (ablation) | ICON · CO3D full-scene | ATE: 0.138 -> 6.215 | Hurt |
| Use confidence calibration of losses | Joint pose and NeRF optimization · CO3D | Bas-Relief solutions: 16% to 9% | Helped |
Surprising findings
- ICON with jointly estimated poses outperforms NeRF trained on CO3D ground-truth (COLMAP) poses on full-scene CO3D.
- COLMAP degrades sharply when restricted to foreground object pixels, while ICON remains accurate.
Limitations
- Evaluated mainly on object-centric data; generalization to scene-level video is only shown on LLFF and full-scene CO3D.
- Restart strategy requires launching multiple independent runs (3), increasing compute.
- Bas-Relief ambiguity is not fully resolved; the confidence calibration only reduces failure rate from 16% to 9%.
Experimental setup
Datasets: CO3D v2 HO3D v2 LLFF
Benchmarks: CO3D v2 full-scene CO3D v2 object-only HO3D v2 LLFF
Metrics: ATE ATErot PSNR SSIM LPIPS Chamfer distance
Baselines: BARF NoPe-NeRF L2G-NeRF LocalRF DROID-SLAM FlowCam PoseDiffusion COLMAP BundleSDF BundleTrack
Builds on: NeRF BARF COLMAP Sampson distance
Closest papers by meaning
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.91)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.89)
- ProvNeRF: Modeling per Point Provenance in NeRFs as a Stochastic Field (NeurIPS 2024 · similarity 0.89)
- AltNeRF: Learning Robust Neural Radiance Field via Alternating Depth-Pose Optimization (AAAI 2024 · similarity 0.89)
- NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.88)
- Improving Robustness for Joint Optimization of Camera Pose and Decomposed Low-Rank Tensorial Radiance Fields (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.