Research map / Image generation / Image restoration
FIRM: Flexible Interactive Reflection ReMoval
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
FIRM is a flexible interactive single-image reflection removal framework that converts points, boxes, strokes, or text guidance into contrastive masks via a SAM-based reflection segmenter (SARM) and removes reflections with a contrastive guidance interaction block in a NAFNet-based network.
Problem
Single-image reflection removal is ambiguous, and existing interactive methods support only one guidance form and need dense user interactions, which makes them slow and impractical.
Approach
Unify heterogeneous user guidance (points, boxes, strokes, text) into a single contrastive mask (reflection=1, transmission=0.5, unannotated=0) using a lightly tuned SAM variant (SARM with a degradation-invariant token and feature selection block), then inject the mask into a reflection removal network via cross-attention in a Contrastive Guidance Interaction Block (CGIB).
Key results
- FIRM with point guidance reaches average PSNR 26.34 and SSIM 0.914 across Real20 and SIR2, above the best single-image (DSRNet 25.46) and interactive (Zhong et al. 25.53) baselines.
- SARM raises reflection-segmentation IOU on synthetic data to 0.429 versus 0.306 for frozen SAM and 0.316 for RobustSAM.
- Guidance takes about 15 seconds per image versus 234 seconds for prior interactive methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use FIRM with point guidance instead of the strongest interactive baseline | reflection removal · Real20 and SIR2 datasets | PSNR: 25.53 -> 26.34 | Helped |
| use FIRM with point guidance instead of the strongest interactive baseline | reflection removal · Real20 and SIR2 datasets | SSIM: 0.909 -> 0.914 | Helped |
| add user guidance rather than using only the blended image (ablation) | FIRM reflection removal network · Real20 and SIR2 average | PSNR: 24.01 -> 26.34 | Helped |
| convert points into contrastive masks with UGC rather than directly combining raw points with the image (ablation) | FIRM reflection removal network · Real20 and SIR2 average | PSNR: 24.05 -> 25.70 | Helped |
| add CGIB deep feature interaction for converted masks (ablation) | FIRM reflection removal network · Real20 and SIR2 average | PSNR: 25.70 -> 26.34 | Helped |
| use contrastive reflection and transmission masks instead of only a reflection mask (ablation) | FIRM reflection removal network · Real20 and SIR2 average | SSIM: 0.902 -> 0.914 | Helped |
| use SARM rather than fine-tuning the entire SAM mask decoder | SAM reflection segmentation · Synthetic Data (COCO validation), reflection | IOU: 0.369 -> 0.429 | Helped |
| use SARM rather than fine-tuning the entire SAM mask decoder | SAM reflection segmentation · SIR2, reflection | IOU: 0.449 -> 0.541 | Helped |
Surprising findings
- Frozen SAM and RobustSAM segment blended images poorly from sparse point prompts, motivating SARM.
- The point-guided FIRM outperforms the text-guided interactive baseline, which the authors attribute to reflections lacking describable semantics.
Limitations
- Text guidance struggles with reflections lacking describable semantics.
- Evaluated only on Real20 and SIR2 benchmarks.
Experimental setup
Datasets: PASCAL VOC Real20 SIR2 COCO
Benchmarks: Real20 SIR2
Metrics: PSNR SSIM IOU Dice
Baselines: Zhang et al. BDN ERRNet IBCLN RAGNet DSRNet FGNet Zhong et al.
Builds on: Segment Anything Model NAFNet RobustSAM
Closest papers by meaning
- Revisiting Single Image Reflection Removal In the Wild (CVPR 2024 · similarity 0.87)
- Flash-Split: 2D Reflection Removal with Flash Cues and Latent Diffusion Separation (CVPR 2025 · similarity 0.85)
- Removing Reflections from RAW Photos (CVPR 2025 · similarity 0.85)
- PolarFree: Polarization-based Reflection-Free Imaging (CVPR 2025 · similarity 0.85)
- Instruct2See: Learning to Remove Any Obstructions Across Distributions (ICML 2025 · similarity 0.84)
- Language-guided Image Reflection Separation (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.