Research map / 3D vision / Depth and stereo
CRAYM: Neural Field Optimization via Camera RAY Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CRAYM, which jointly optimizes camera poses and a hash-encoded feature volume (neural field) using key-ray matching, keypoint-based matched-ray coherence, and epipolar and point-alignment losses for novel view synthesis and SDF-based reconstruction from noisy-pose multi-view images.
Problem
Joint optimization of camera poses and neural fields from multi-view images is unstable when input camera poses are noisy, leading to floaters, blur, and failed reconstructions.
Approach
Matches camera rays (not pixels) through SuperPoint/SuperGlue keypoints and parameterizes the rays by a feature volume, so matched-ray color coherence is weighted by a learned matchability (cosine similarity of accumulated ray features), plus epipolar and point-alignment geometric losses and auxiliary-ray feature enrichment.
Key results
- On NeRF-Synthetic with noisy poses, CRAYM achieves mean PSNR 30.34 and CD 0.06 vs. L2G-NeRF 28.62 and 0.17.
- On LLFF, CRAYM reaches mean PSNR 24.76 and LPIPS 0.18, above L2G-NeRF (24.54, 0.20) and BARF (24.23, 0.23).
- On UrbanScene3D PolyTech, CRAYM reports PSNR 22.51 vs. NeuS 14.27 and PET-NeuS 13.64.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add KRE to the progressive feature-mask baseline (ablation) | CRAYM · LEGO high noise | PSNR: 27.30 -> 28.64 | Helped |
| Add MRC to the KRE variant (ablation) | CRAYM · LEGO high noise | PSNR: 28.64 -> 30.41 | Helped |
| Add epipolar loss L_e to the baseline (ablation) | CRAYM · LEGO high noise | PSNR: 27.30 -> 29.43 | Helped |
| Add point-alignment loss L_a to the L_e variant (ablation) | CRAYM · LEGO high noise | PSNR: 29.43 -> 29.95 | Helped |
| Use the full pipeline instead of the KRE + MRC variant (ablation) | CRAYM · LEGO high noise | PSNR: 30.41 -> 31.60 | Helped |
| Use CRAYM instead of L2G-NeRF | NeRF-Synthetic mean (8 objects) · NeRF-Synthetic mean (8 objects) | PSNR: 28.62 -> 30.34 | Helped |
| Use CRAYM instead of L2G-NeRF | LLFF · LLFF mean | LPIPS: 0.20 -> 0.18 | Helped |
| Use CRAYM instead of SPARF | LEGO sparse input · LEGO (3-view sparse input) | LPIPS: 0.40 -> 0.41 | Hurt |
Surprising findings
- SPARF-style sparse-view comparison shows CRAYM is only comparable to SPARF (PSNR 16.08 vs 15.91) on 3-view LEGO, despite being designed for dense views.
- On LLFF, CRAYM is not best on every scene; BAA-NGP wins on Flower, Leaves, and T-rex for some metrics.
Limitations
- Pose optimization may not converge when poses are randomly initialized or extremely noisy.
- Extracted SDF meshes may contain messy inner structures over invisible areas.
- Relies on sparse key rays, so dense-view use may add overhead; sparse-view performance is comparable rather than superior.
Experimental setup
Datasets: NeRF-Synthetic LLFF UrbanScene3D DTU TwinTex
Benchmarks: NeRF-Synthetic LLFF UrbanScene3D DTU TwinTex
Metrics: PSNR SSIM LPIPS Chamfer distance Hausdorff distance precision recall F-score rotation error translation error
Baselines: NeRF NeuS BARF L2G-NeRF SPARF PET-NeuS BAA-NGP NeuS2
Builds on: NeuS NeRF BARF NeuS2 SuperGlue
Closest papers by meaning
- Cameras as Rays: Pose Estimation via Ray Diffusion (ICLR 2024 · similarity 0.86)
- LEAP: Liberate Sparse-View 3D Modeling from Camera Poses (ICLR 2024 · similarity 0.85)
- NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.85)
- Improving Robustness for Joint Optimization of Camera Pose and Decomposed Low-Rank Tensorial Radiance Fields (AAAI 2024 · similarity 0.85)
- Structure Consistent Gaussian Splatting with Matching Prior for Few-shot Novel View Synthesis (NeurIPS 2024 · similarity 0.85)
- Joint Optimization of Neural Radiance Fields and Continuous Camera Motion from a Monocular Video (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.