Research map / 3D vision / Depth and stereo
Passive Snapshot Coded Aperture Dual-Pixel RGB-D Imaging
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CADS, a learned amplitude coded aperture placed in a dual-pixel camera lens, jointly optimized end-to-end with a U-Net-style network (CADNet) to recover depth and all-in-focus images in a single snapshot.
Problem
Dual-pixel sensors give compact passive single-shot depth, but disparity is proportional to defocus blur, creating a trade-off between disparity estimation accuracy and all-in-focus deblurring quality.
Approach
Insert a learned amplitude mask in the aperture plane of a dual-pixel camera so that DP defocus PSFs take the mask shape (Hadamard product with naive DP PSFs), better conditioning deblurring while preserving disparity; mask and CADNet are trained end-to-end on a differentiable coded DP image simulator.
Key results
- CADS achieves 31.20 dB AIF PSNR vs 29.72 dB for naive DP in simulation (f/4), with depth MAE 5.15 mm vs 5.51 mm.
- Across f/4 to f/10, CADS improves AIF PSNR by about 1.5 dB over naive DP at each aperture (e.g., 34.10 vs 32.68 dB at f/10).
- On Table 3 comparison, CADS obtains AI(1) disparity error 0.0177 vs 0.0190 for naive DP and 0.1940-0.3018 for prior DP methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the learned amplitude aperture mask and end-to-end optimization | Naive DP · FlyingThings3D, 20 cm depth range, f/4 | AIF PSNR (dB): 29.72 -> 31.20 | Helped |
| Add the learned amplitude aperture mask and end-to-end optimization | Naive DP · FlyingThings3D, 20 cm depth range, f/4 | depth MAE (mm): 5.51 -> 5.15 | Helped |
| Add the learned amplitude aperture mask and end-to-end optimization | Naive DP · FlyingThings3D, f/4 to f/10 | AIF PSNR (dB): 29.54 -> 31.21; 31.07 -> 32.66; 31.98 -> 33.49; 32.68 -> 34.10 | Helped |
| Add the learned amplitude aperture mask and end-to-end optimization | Naive DP · FlyingThings3D, f/4 to f/10 | depth MAE (mm): 5.44 -> 4.99; 5.48 -> 5.12; 5.65 -> 5.46; 6.19 -> 6.03 | Helped |
| Use CADS instead of DPDNet | DPDNet · FlyingThings3D validation subset | AIF PSNR: 24.34 -> 31.20 | Helped |
| Use CADS instead of Punnappurath et al. | Punnappurath et al. · FlyingThings3D validation subset | Disp. AI(1): 0.1940 -> 0.0177 | Helped |
Surprising findings
- Existing unidirectional-disparity DP methods (DDDNet, Xin et al.) perform poorly on bidirectional-disparity simulated scenes.
Limitations
- Amplitude mask reduces light efficiency (constrained to at least 50% transmission), lowering SNR.
- Real-world results rely on fine-tuning with captured PSFs and spatially varying PSFs in endoscopy and dermoscopy.
- Endoscopy and dermoscopy results are qualitative only, with coarse ground truth depth from RealSense.
Experimental setup
Datasets: FlyingThings3D NYUv2
Benchmarks: FlyingThings3D validation NYUv2 rendered images real Canon EOS 5D Mark IV captures
Metrics: PSNR SSIM LPIPS RMSE MAE delta1 affine-invariant MAE
Baselines: Naive Standard-Pixel Naive Dual-Pixel Coded SP DPDNet DDDNet Xin et al. Punnappurath et al. Kim et al.
Builds on: Levin et al. coded aperture depth Abuolaim DPDNet Ikoma differentiable occlusion-aware rendering U-Net Shedligeri end-to-end coded mask
Closest papers by meaning
- Focal Split: Untethered Snapshot Depth from Differential Defocus (CVPR 2025 · similarity 0.84)
- Learned Binocular-Encoding Optics for RGBD Imaging Using Joint Stereo and Focus Cues (CVPR 2025 · similarity 0.83)
- Quad-Pixel Image Defocus Deblurring: A New Benchmark and Model (CVPR 2025 · similarity 0.83)
- LDP: Language-driven Dual-Pixel Image Defocus Deblurring Network (CVPR 2024 · similarity 0.82)
- Time-Efficient Light-Field Acquisition Using Coded Aperture and Events (CVPR 2024 · similarity 0.82)
- All-directional Disparity Estimation for Real-world QPD Images (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.