Research map / Image generation / Image restoration
Removing Reflections from RAW Photos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains a RAW-based reflection removal system solely on photometrically and geometrically accurate synthetic mixtures of real RAW photos, with an optional contextual photo and a learned upsampler from 256p to full resolution.
Problem
Removing real-world reflections from consumer photos through glass, where prior methods use tone-mapped 8-bit images at low resolution, need ground-truth real pairs, and produce inaccurate color.
Approach
Simulate reflections by linearly summing RAW-derived XYZ images before white balancing (preserving illuminant colors and luminance), use an optional context photo of the reflected scene to disambiguate reflection, and upsample via feature-matched masks applied to the high-res mixture rather than to upsampled outputs.
Key results
- Ours+ctx reaches SSIM_t 95.7% and SSIM_r 88.2% on the synthetic RAW test set (PSNR_t 33.23), versus 89.9% SSIM_t for Zhang et al. retrained on RAW.
- Retraining prior models on RAW raises SSIM_t by about 40 points, larger than the ~20-point spread among architectures, and ablating the simulation drops performance by 46 points.
- The upsampler reaches PSNR_t 30.62 end-to-end from 256p to 2048p, outperforming V-DESIRR (30.27), and the base model runs in about 1s at 256x341 on an M1 MacBook Pro or iPhone 14 Pro.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use full photometrically and geometrically accurate RAW reflection simulation instead of the control data (ablation) | Base model · Synthetic RAW test set | SSIMt: 85.8 -> 95.7 | Helped |
| Gamma-compress transmission and reflection before compositing (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 88.8 | Hurt |
| Expose transmission and reflection separately (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 94.2 | Hurt |
| Remove constraints on inclination and field of view (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 95.0 | Hurt |
| Remove spatially varying Fresnel attenuation (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 95.3 | Hurt |
| White-balance transmission and reflection separately (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 95.5 | Hurt |
| Remove depth-of-field blur and double reflection (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 95.5 | Hurt |
| Use all simulation ablations together (ablation) | Base model trained on simulated data · Synthetic RAW test set | SSIMt: 95.7 -> 89.2 | Hurt |
| Add contextual photo input (ablation) | Reflection removal model · Synthetic RAW test set | SSIMt: 95.1 -> 95.7 | Helped |
| Replace real context with random context (ablation) | Contextual reflection removal model · Synthetic RAW test set | SSIMt: 95.7 -> 95.1 | Hurt |
| Use the ground-truth reflection as context at test time (ablation) | Contextual reflection removal model · Synthetic RAW test set | SSIMt: 95.7 -> 95.7 | No clear effect |
| Remove upsampler masking operations (ablation) | Upsampler · Upsampling GT images, 256p to 2048p | PSNRt: 47.77 -> 43.29 | Hurt |
Surprising findings
- Retraining prior architectures on simulated RAW data gives larger gains than architectural variations among prior methods.
- Previously published 8-bit models (Zhang, CoRRN) distort color on real photos, with CoRRN's reflection SSIM dropping below its control.
Limitations
- Fails when transmission or reflection is bright and pushes the other into the noise floor, causing hole-filling problems.
- Errors occur when textured regions of transmission and reflection overlap, with no color difference to separate them.
- Upsampler errors when features absent at low resolution become visible at higher levels.
Experimental setup
Datasets: MIT-Adobe FiveK (MIT5K) RAISE Laval Indoor HDR Synthetic reflection mixtures from RAW images
Benchmarks: Synthetic RAW test set (5% held-out simulation split) In-the-wild captured RAW photos (qualitative) Ground-truth captured reflection examples
Metrics: PSNR SSIM
Baselines: Zhang et al. DSRNet Zhu et al. CoRRN V-DESIRR SUPIR RESRGAN
Builds on: Zhang et al. reflection removal (CVPR 2019 style losses) EfficientNet BiFPN V-DESIRR
Closest papers by meaning
- Revisiting Single Image Reflection Removal In the Wild (CVPR 2024 · similarity 0.87)
- FIRM: Flexible Interactive Reflection ReMoval (AAAI 2025 · similarity 0.85)
- MirrorVerse: Pushing Diffusion Models to Realistically Reflect the World (CVPR 2025 · similarity 0.84)
- Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation (CVPR 2025 · similarity 0.84)
- ReFIR: Grounding Large Restoration Models with Retrieval Augmentation (NeurIPS 2024 · similarity 0.83)
- Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.