Research map / 3D vision / NeRF and radiance fields

CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs

CVPR 2024 · 3D vision › NeRF and radiance fields · approach: Dynamic and editable NeRF · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adds a voxel-based ray sampling scheme, a light-weight In-Voxel Transformer, and a voxel contrastive loss to NeRF-style radiance fields to enforce 3D spatial field consistency under sparse input views.

Problem

Novel view synthesis with NeRF from sparse input views suffers from incorrect density and color predictions due to partial and sparse supervision; prior regularizers act only at the ray or 2D image level, not 3D spatial field consistency.

Approach

Sample rays per voxel so that points inside a voxel share radiance; a Transformer encoder-decoder predicts ray-point colors and densities from randomly sampled surrounding points in the same voxel (local implicit constraint), and a contrastive loss pulls same-voxel region features together and pushes other-voxel features apart (global explicit constraint). The module is plugged into NeRF, BARF, SP

Key results

• On DTU 3-view, CVT-xRF (w/ BARF) reaches 18.06 full-image PSNR vs 16.43 for BARF, and CVT-xRF (w/ NeRF) improves NeRF by 7.45 PSNR on DTU 3-view.
• CVT-xRF (w/ SPARF) reaches 18.98 full-image PSNR on DTU 3-view vs 18.32 for SPARF, and 27.04 vs 25.75 at 9 views.
• On Synthetic 3-view, CVT-xRF (w/ BARF) reaches 21.58 PSNR and 0.851 SSIM, outperforming listed baselines except none exceed it in that table.

Measured effects reported in the paper

Surprising findings

• Ablation shows the voxel-based ray sampling alone, without any consistency loss, already improves NeRF on DTU 3-view (6.68 to 7.23 PSNR).

Limitations

• On the Synthetic 8-view setting, CVT-xRF (w/ BARF) underperforms, so the method needed FreeNeRF's occlusion regularization to be competitive.
• Gains are reported mainly against weaker or regularized baselines; the method is comparable to SPARF rather than clearly superior at 6/9 views on DTU.

Experimental setup

Datasets: DTU Synthetic (NeRF blender)

Benchmarks: DTU Synthetic NeRF dataset

Metrics: PSNR SSIM

Baselines: NeRF BARF SPARF RegNeRF FreeNeRF DietNeRF PixelNeRF MVSNeRF mip-NeRF FlipNeRF

Builds on: NeRF BARF SPARF FreeNeRF SimCLR

Closest papers by meaning

• Sparse Voxels Rasterization: Real-time High-fidelity Radiance Field Rendering (CVPR 2025 · similarity 0.90)
• NeRF-VPT: Learning Novel View Representations with Neural Radiance Fields via View Prompt Tuning (AAAI 2024 · similarity 0.89)
• Learning with Unreliability: Fast Few-shot Voxel Radiance Fields with Relative Geometric Consistency (CVPR 2024 · similarity 0.89)
• Rad-NeRF: Ray-decoupled Training of Neural Radiance Field (NeurIPS 2024 · similarity 0.89)
• NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.88)
• GoLF-NRT: Integrating Global Context and Local Geometry for Few-Shot View Synthesis (CVPR 2025 · similarity 0.88)