Research map / Architectures and efficiency / Attention and transformers

Single Image Reflection Separation via Dual-Stream Interactive Transformers

NeurIPS 2024 · Architectures and efficiency › Attention and transformers · approach: Efficient image transformers · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DSIT, a dual-stream interactive Transformer for single image reflection separation that uses dual-attention (self- and cross-attention over both layers) and cross-architecture interaction with a pre-trained Swin-L prior to beat prior dual-stream methods on real-world benchmarks.

Problem

Single image reflection separation fails on complex cases where the transmission layer is densely entangled with a spatially widely distributed reflection; prior dual-stream interaction does not explicitly assess feature correlation and has limited receptive field.

Approach

Dual-attention interactive block combining dual-stream self-attention (intra-layer) and layer-aware dual-stream cross-attention (inter-layer, with layered relative position bias) to explicitly model feature correlations, plus a dual-architecture encoder that modulates dual-stream CNN features with global priors from a pre-trained Swin Transformer via cross-architecture interactions.

Key results

• DSIT achieves average PSNR 26.27 / SSIM 0.917 across four real-world datasets, versus 25.40 / 0.905 for DSRNet under setting I.
• Under data setting II, DSIT reaches 26.49 PSNR / 0.922 SSIM on the average of four datasets, beating DSRNet (25.75 / 0.910) and RRW (25.34 / 0.912).
• On the Nature test set, DSIT reaches 26.77 PSNR / 0.847 SSIM under setting II, compared with 25.22 / 0.832 for DSRNet.

Measured effects reported in the paper

Surprising findings

• Cross-attention-based cross-architecture fusion performed worse than simple addition (24.50 vs 24.62 PSNR on Real20), so plain cross-attention alone was not enough.

Limitations

• Regions dominated by reflections with negligible defocus remain partly unseparated, violating the relative smoothness assumption.
• Gains depend on the choice of global prior extractor; ResNet101 and PVTv2 priors generalize worse.
• No error bars reported.

Experimental setup

Datasets: PASCAL VOC Real20 Nature SIR2

Benchmarks: Real20 Objects Postcard Wild Nature SIR2

Metrics: PSNR SSIM

Baselines: Zhang et al. BDN ERRNet IBCLN RAGNet DMGN Zheng et al. YTMT RobustSIRR DSRNet PromptRR Dong et al.

Builds on: MuGI DSRNet YTMT Swin Transformer ERRNet

Closest papers by meaning

• CARE Transformer: Mobile-Friendly Linear Visual Transformer via Decoupled Dual Interaction (CVPR 2025 · similarity 0.85)
• Language-guided Image Reflection Separation (CVPR 2024 · similarity 0.85)
• Linear Differential Vision Transformer: Learning Visual Contrasts via Pairwise Differentials (NeurIPS 2025 · similarity 0.84)
• Reversible Decoupling Network for Single Image Reflection Removal (CVPR 2025 · similarity 0.84)
• Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion (CVPR 2025 · similarity 0.84)
• CLEAR: Conv-Like Linearization Revs Pre-Trained Diffusion Transformers Up (NeurIPS 2025 · similarity 0.83)