Research map / 3D vision / Depth and stereo
Dr. Bokeh: DiffeRentiable Occlusion-aware Bokeh Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dr.Bokeh, a non-learning, differentiable, occlusion-aware filtering-based bokeh renderer that models on-focal and non-focal occlusion over layered RGBA-D scenes, and shows gains on synthetic/real bokeh rendering and depth-from-defocus.
Problem
Filtering-based bokeh rendering ignores occlusion, causing color bleeding and unnatural partial-occlusion artifacts; learning-based fixes are not physically grounded and fail to generalize.
Approach
Extends the scattering/gathering bokeh equation with on-focal occlusion terms (soft binary occlusion per layer) and non-focal visibility terms that blend layers front-to-back, yielding a physically motivated, differentiable renderer that needs no training and can plug into depth-from-defocus pipelines.
Key results
- On the synthetic benchmark Dr.Bokeh achieves RMSE 0.0133, PSNR 38.73, SSIM 0.9757, ZNCC 0.9979, best among SteReFo, DeepLens, BokehMe, MPIB.
- In a 41-person 2AFC user study, 81%, 74%, 68%, 61% preferred Dr.Bokeh over SteReFo, DeepLens, BokehMe, MPIB for realism.
- In depth-from-defocus on the Light Field Dataset, Dr.Bokeh reaches RMSE 0.0123 and SSIM 0.9807 vs 0.0133/0.9774 for Aperture, and an ablation shows removing occlusion raises RMSE to 0.0139.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use occlusion-aware Dr.Bokeh rendering instead of prior bokeh renderers | bokeh rendering · Synthetic benchmark | RMSE: 0.0144 -> 0.0133 | Helped |
| Use occlusion-aware Dr.Bokeh rendering instead of prior bokeh renderers | bokeh rendering · Synthetic benchmark | PSNR: 37.77 -> 38.73 | Helped |
| Add the occlusion term to the differentiable renderer (ablation) | Dr.Bokeh depth-from-defocus training · Light Field Dataset | RMSE: 0.0139 -> 0.0123 | Helped |
| Add hierarchy SSIM loss instead of L1 + Grad loss (ablation) | Dr.Bokeh depth-from-defocus training · Light Field Dataset | RMSE: 0.0146 -> 0.0123 | Helped |
| Use hierarchy SSIM loss instead of ordinary SSIM loss (ablation) | Dr.Bokeh depth-from-defocus training · Light Field Dataset | SSIM: 0.9740 -> 0.9807 | Helped |
Surprising findings
- The fully learning-based MPIB underperforms the hybrid BokehMe on quantitative metrics, attributed to poor generalization, despite producing more natural partial occlusion qualitatively.
Limitations
- Inaccurate input depth and inpainting produce artifacts.
- Relies on off-the-shelf salient detection and inpainting (LaMa) for layered representation.
Experimental setup
Datasets: Adobe Matting Dataset AIM-500 Landmark dataset Light Field Dataset
Benchmarks: Synthetic ray-traced bokeh benchmark (100 scenes) Light Field Dataset
Metrics: RMSE RMSE-s PSNR SSIM ZNCC user preference (2AFC)
Baselines: SteReFo DeepLens BokehMe MPIB Aperture GaussPSF
Builds on: SteReFo Aperture supervision GaussPSF LaMa inpainting BokehMe
Closest papers by meaning
- AnyBokeh: Physics-Guided Any-to-Any Bokeh Editing with Optical Fingerprint Transfer (NeurIPS 2026 · similarity 0.87)
- BLURD: Benchmarking and Learning using a Unified Rendering and Diffusion Model (NeurIPS 2024 · similarity 0.83)
- CoCoGaussian: Leveraging Circle of Confusion for Gaussian Splatting from Defocused Images (CVPR 2025 · similarity 0.82)
- UniGS: Unified Geometry-Aware Gaussian Splatting for Multimodal Rendering (NeurIPS 2026 · similarity 0.82)
- NeRF On-the-go: Exploiting Uncertainty for Distractor-free NeRFs in the Wild (CVPR 2024 · similarity 0.82)
- UMAMI: Unifying Masked Autoregressive Models and Deterministic Rendering for View Synthesis (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.