Research map / Vision-language and multimodal / Cross-modal learning
Language-guided Image Reflection Separation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes language-guided image reflection separation, using a CLIP-text-conditioned gated cross-attention network with contrastive correspondence losses, trained on captioned synthetic and real reflection data.
Problem
Single-image reflection separation is highly ill-posed; existing methods lack content priors about the transmission and reflection layers, and no datasets pair reflection images with language descriptions.
Approach
Use natural-language descriptions of one or both layers to guide separation via adaptive global aggregation/interaction modules with channel-wise cross-attention, a language gate that handles missing descriptions, and contrastive and layer correspondence losses that align text with the correct recovered layer.
Key results
- Achieves average PSNR 25.72 and SSIM 0.914 on transmission layers across five real subsets, versus 25.51/0.906 for the best baseline DSRNet.
- Ablation: removing language drops PSNR from 25.72 to 24.31; replacing AGIM gives 24.69.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add language descriptions (ablation) | Language-guided reflection separation · Average (540) | PSNR: 24.31 -> 25.72 | Helped |
| add language descriptions (ablation) | Language-guided reflection separation · Average (540) | SSIM: 0.885 -> 0.914 | Helped |
| use AGIM instead of concatenation and self-attention blocks (ablation) | Language-guided reflection separation · Average (540) | PSNR: 24.69 -> 25.72 | Helped |
| use AGIM instead of concatenation and self-attention blocks (ablation) | Language-guided reflection separation · Average (540) | SSIM: 0.901 -> 0.914 | Helped |
| add cross-modality correspondence losses to Limg (ablation) | Language-guided reflection separation · Average (540) | PSNR: 24.52 -> 25.72 | Helped |
| add cross-modality correspondence losses to Limg (ablation) | Language-guided reflection separation · Average (540) | SSIM: 0.893 -> 0.914 | Helped |
| use language-guided reflection separation instead of DSRNet | Reflection separation · Average (540) | PSNR: 25.51 -> 25.72 | Helped |
| use language-guided reflection separation instead of DSRNet | Reflection separation · Average (540) | SSIM: 0.906 -> 0.914 | Helped |
Limitations
- May fail to separate transmission and reflection layers when their contents are similar.
- Evaluation only on real datasets with manually written descriptions; REFOL dataset not stated as released.
Experimental setup
Datasets: Flickr30k COCO Captions Nature Zhang et al. real dataset SIR2
Benchmarks: Nature Real20 SIR2 (Postcard, Object, Wild subsets) REFOL
Metrics: PSNR SSIM
Baselines: DSRNet YTMT Dong et al. IBCLN CoRRN Zhang et al.
Builds on: CLIP DSRNet IBCLN Restormer-style LeFF blocks
Closest papers by meaning
- Single Image Reflection Separation With Perceptual Losses (CVPR 2018 · similarity 0.87)
- ReflexSplit: Single Image Reflection Separation via Layer Fusion-Separation (CVPR 2026 · similarity 0.87)
- Single Image Reflection Separation via Dual-Stream Interactive Transformers (NeurIPS 2024 · similarity 0.85)
- FIRM: Flexible Interactive Reflection ReMoval (AAAI 2025 · similarity 0.84)
- FFF: Fixing Flawed Foundations in Contrastive Pre-Training Results in Very Strong Vision-Language Models (CVPR 2024 · similarity 0.83)
- Enhancing Multi-Image Understanding through Delimiter Token Scaling (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.