Research map / 3D vision / NeRF and radiance fields
SCINeRF: Neural Radiance Fields from a Snapshot Compressive Image
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SCINeRF, which recovers a NeRF 3D scene and camera poses from a single snapshot compressed (SCI) image by optimizing NeRF and poses through the differentiable SCI forward model.
Problem
Recovering multi-view consistent high frame-rate 3D scene content from a single temporally compressed snapshot image, where existing SCI decoders reconstruct only the mask-encoded frames and ignore 3D structure and camera poses are unavailable.
Approach
Embed the SCI image formation model (binary-mask-modulated sum of virtual frames) directly into NeRF training, jointly optimizing NeRF weights and per-frame camera poses (linearly interpolated start/end SE(3) poses) via a photometric loss on the single compressed measurement, enabling test-time optimization without learned priors.
Key results
- On synthetic scenes at compression ratio 8, SCINeRF reaches e.g. 33.23 PSNR on Cozy2room and 36.60 on Factory versus 31.47 and 32.87 for EfficientSCI, outperforming SCI baselines.
- For novel view synthesis, SCINeRF achieves 30.61 PSNR on Airplants versus 28.62 for NeRF+EfficientSCI, with the unfair advantage of GT-derived poses given to baselines.
- At compression ratios 16, 24, 32 SCINeRF keeps PSNR around 34.9/34.4/34.0 while EfficientSCI drops to 31.6/29.1/28.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Airplants | PSNR: 30.13 -> 30.69 | Helped |
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Hotdog | PSNR: 30.75 -> 31.35 | Helped |
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Cozy2room | PSNR: 31.47 -> 33.23 | Helped |
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Tanabata | PSNR: 32.30 -> 33.61 | Helped |
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Factory | PSNR: 32.87 -> 36.60 | Helped |
| Use SCINeRF instead of EfficientSCI for SCI image reconstruction | SCI image reconstruction · Vender | PSNR: 33.17 -> 36.40 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Airplants | PSNR: 28.62 -> 30.61 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Hotdog | PSNR: 29.82 -> 30.59 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Cozy2room | PSNR: 31.79 -> 33.12 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Tanabata | PSNR: 30.79 -> 32.84 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Factory | PSNR: 32.56 -> 36.33 | Helped |
| Use SCINeRF instead of NeRF with reconstructed SCI images for novel-view synthesis | Novel-view synthesis · Vender | PSNR: 32.77 -> 34.75 | Helped |
Surprising findings
- SSIM of SCINeRF is lower than EfficientSCI on some scenes (e.g., Airplants 0.9335 vs 0.9425), attributed to rendering rather than direct measurement recovery.
Limitations
- Test-time optimization per scene; requires a single scene's measurement and masks, with no generalization.
- Assumes linear camera trajectory within exposure for pose interpolation; more complex motion needs higher-order splines or per-pose optimization.
- Baselines were given GT-derived poses, and rendering can lose fine image detail (lower SSIM in some scenes).
Experimental setup
Datasets: LLFF NeRF Synthetic360 DeblurNeRF Blender scenes real SCI measurements captured with custom system
Benchmarks: LLFF Airplants NeRF Synthetic Hotdog DeblurNeRF Blender scenes (Cozy2room, Tanabata, Factory, Vender) real SCI dataset
Metrics: PSNR SSIM LPIPS
Baselines: GAP-TV PnP-FFDNet PnP-FastDVDNet EfficientSCI NeRF+GAP-TV NeRF+PnP-FFDNet NeRF+PnP-FastDVDNet NeRF+EfficientSCI
Builds on: NeRF BARF BAD-NeRF EfficientSCI GAP-TV
Closest papers by meaning
- Neural NeRF Compression (ICML 2024 · similarity 0.89)
- Untrained Neural Nets for Snapshot Compressive Imaging: Theory and Algorithms (NeurIPS 2024 · similarity 0.88)
- ColNeRF: Collaboration for Generalizable Sparse Input Neural Radiance Field (AAAI 2024 · similarity 0.88)
- FrugalNeRF: Fast Convergence for Extreme Few-shot Novel View Synthesis without Learned Priors (CVPR 2025 · similarity 0.88)
- CF-NeRF: Camera Parameter Free Neural Radiance Fields with Incremental Learning (AAAI 2024 · similarity 0.87)
- TeTriRF: Temporal Tri-Plane Radiance Fields for Efficient Free-Viewpoint Video (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.