Research map / 3D vision / NeRF and radiance fields
Structure-Aware Sparse-View X-ray 3D Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAX-NeRF, a Transformer-based X-ray neural radiodensity field with line-segment self-attention and masked local-global ray sampling, plus a new 15-scene X3D dataset, for sparse-view X-ray novel view synthesis and CT reconstruction.
Problem
Sparse-view X-ray 3D reconstruction (novel view synthesis and CT reconstruction) with NeRF-style methods, which ignore X-ray's attenuation/internal-structure nature and use MLPs and naive pixel-level ray sampling.
Approach
Replace the MLP radiodensity field with a Line Segment-based Transformer that computes self-attention within each segment of an X-ray (linear cost in points), and sample rays with a masked local-global strategy that combines patch-level and pixel-level sampling on foreground regions.
Key results
- On X3D novel view synthesis, SAX-NeRF averages 51.37 dB PSNR / 0.9994 SSIM vs. 38.81 dB for NAF and 37.67 dB for TensoRF.
- On X3D CT reconstruction, SAX-NeRF averages 37.25 dB PSNR / 0.9753 SSIM, 2.49 dB above NAF (34.76 dB).
- Ablation: LS-MSA and MLG together add 13.40 dB (NVS) and 3.04 dB (CT) over the baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LS-MSA to the baseline (ablation) | SAX-NeRF · X3D NVS average | PSNR: 37.97 -> 47.95 | Helped |
| Add LS-MSA to the baseline (ablation) | SAX-NeRF · X3D CT reconstruction average | PSNR: 34.21 -> 36.86 | Helped |
| Add MLG sampling to the baseline (ablation) | SAX-NeRF · X3D NVS average | PSNR: 37.97 -> 43.51 | Helped |
| Add MLG sampling to the baseline (ablation) | SAX-NeRF · X3D CT reconstruction average | PSNR: 34.21 -> 35.30 | Helped |
| Add LS-MSA and MLG sampling (ablation) | SAX-NeRF · X3D NVS average | PSNR: 37.97 -> 51.37 | Helped |
| Add LS-MSA and MLG sampling (ablation) | SAX-NeRF · X3D CT reconstruction average | PSNR: 34.21 -> 37.25 | Helped |
| Set line segment quantity to 20 (ablation) | SAX-NeRF · X3D NVS average | PSNR: 51.365 -> 45.776 | Hurt |
| Set line segment quantity to 40 (ablation) | SAX-NeRF · X3D NVS average | PSNR: 51.365 -> 47.832 | Hurt |
| Set line segment quantity to 80 (ablation) | SAX-NeRF · X3D NVS average | PSNR: 51.365 -> 50.148 | Hurt |
| Set line segment quantity to 320 (ablation) | SAX-NeRF · X3D NVS average | PSNR: 51.365 -> 50.620 | Hurt |
| Replace LS-MSA with vanilla Transformer G-MSA (ablation) | SAX-NeRF · X3D NVS average | PSNR: +5.30 dB | Hurt |
| Replace LS-MSA with vanilla Transformer G-MSA (ablation) | SAX-NeRF · X3D CT reconstruction average | PSNR: +1.28 dB | Hurt |
Surprising findings
- Using only 60% of training projections, SAX-NeRF still outperforms other methods on NVS.
Limitations
- Evaluated only on simulated projections generated from CT volumes with 3% noise, with 50 training views per scene.
- Requires per-scene training and does not use CT pairs, so generalization across scenes is not studied.
Experimental setup
Datasets: X3D LIDC-IDRI MIDA D2VR DeepOrgan VOLVIS open scientific visualization dataset
Benchmarks: X3D
Metrics: PSNR SSIM
Baselines: InTomo NeRF NeAT TensoRF NAF FDK ASD-POCS SART
Builds on: NeRF TensoRF NAF Vision Transformer
Closest papers by meaning
- CVT-xRF: Contrastive In-Voxel Transformer for 3D Consistent Radiance Fields from Sparse Inputs (CVPR 2024 · similarity 0.85)
- X-Ray: A Sequential 3D Representation For Generation (NeurIPS 2024 · similarity 0.84)
- Structure from Collision (CVPR 2025 · similarity 0.84)
- GO-N3RDet: Geometry Optimized NeRF-enhanced 3D Object Detector (CVPR 2025 · similarity 0.84)
- Sparse3D: Distilling Multiview-Consistent Diffusion for Object Reconstruction from Sparse Views (AAAI 2024 · similarity 0.84)
- PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.