Research map / Architectures and efficiency / Attention and transformers
Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TBSN, a transformer-based blind-spot network for self-supervised real-world image denoising with grouped channel attention and masked window attention, plus a distillation step into a plain U-Net (TBSN2UNet).
Problem
Transformers could improve blind-spot networks (BSN) for self-supervised image denoising, but their attention mechanisms may violate the blind-spot requirement and leak the target pixel, causing overfitting to noisy inputs.
Approach
Redesign channel self-attention by splitting channels into groups so each group stays smaller than spatial resolution (avoiding blind-spot leakage in multi-scale U-Nets), and redesign window self-attention with a fixed parity mask that mimics dilated convolution receptive fields; then distill the resulting TBSN into a plain U-Net for cheap inference.
Key results
- TBSN reaches 37.78 dB PSNR on SIDD benchmark and 39.08 dB on DND, outperforming AT-BSN and other self-supervised methods on DND (PSNR).
- TBSN2UNet, a 1.08M-parameter U-Net distilled from TBSN, achieves 37.79 dB on SIDD benchmark with 35.0 GFLOPs at 256x256.
- Dilated G-CSA with 48 channels per group improves over plain dilated CSA at deep 4- and 5-scale levels, where plain CSA drops to 36.84 and 11.35 dB on SIDD validation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dilated M-WSA to the dilated-convolution base (ablation) | TBSN · SIDD validation | PSNR: +0.27dB | Helped |
| Add dilated G-CSA to the dilated-convolution base (ablation) | TBSN · SIDD validation | PSNR: +0.65dB | Helped |
| Replace dilated M-WSA with SwinIA (ablation) | TBSN · SIDD validation | PSNR: 0.59dB performance drop | Hurt |
| Replace dilated G-CSA with LG-BPN channel attention (ablation) | TBSN · SIDD validation | PSNR: 0.05dB performance drop | Hurt |
| Replace dilated M-WSA with Swin Transformer (ablation) | TBSN · SIDD validation | PSNR: 37.71 -> 37.56 | Hurt |
| Replace dilated G-CSA with SE (ablation) | TBSN · SIDD validation | PSNR: 37.71 -> 37.43 | Hurt |
| Replace dilated G-CSA with SCA (ablation) | TBSN · SIDD validation | PSNR: 37.71 -> 37.63 | Hurt |
| Use ungrouped dilated CSA at 4- and 5-scale architectures (ablation) | Multi-scale blind-spot network · SIDD validation | PSNR: 37.07 -> 36.84; 37.02 -> 11.35 | Hurt |
| Group dilated CSA channels to prevent blind-spot leakage (ablation) | Multi-scale blind-spot network · SIDD validation | PSNR: 36.84 -> 37.38; 11.35 -> 37.31 | Helped |
| Distill TBSN into a U-Net student (ablation) | TBSN · SIDD validation | PSNR: 37.71 -> 37.70 | No clear effect |
| Use supervised training for U-Net instead of knowledge distillation (ablation) | U-Net · SIDD validation | PSNR: 37.70 -> 38.92 | Helped |
| Distill TBSN into a smaller U-Net | TBSN · SIDD Bench | PSNR: +0.01dB | Helped |
Surprising findings
- Plain dilated channel self-attention at the 5-scale architecture collapses to 11.35 dB on SIDD validation, indicating severe blind-spot leakage.
- A supervised U-Net trained on clean pairs outperforms TBSN by 1.21 dB on SIDD validation, suggesting lightweight networks can match complex self-supervised ones.
Limitations
- TBSN has substantially higher inference cost (about 1032 ms at 256x256) than the distilled student.
- Real-world evaluation relies on SIDD and DND only, with DND trained on its test images in a self-supervised manner.
Experimental setup
Datasets: SIDD DND
Benchmarks: SIDD benchmark DND benchmark SIDD validation
Metrics: PSNR SSIM FLOPs parameters inference time
Baselines: Noise2Void Noise2Self NAC R2R CVF-SID AP-BSN SASL LG-BPN PUCA AT-BSN
Builds on: Blind-spot network Restormer (MDTA) AP-BSN Dilated BSN Swin Transformer
Closest papers by meaning
- Pseudo-Siamese Blind-spot Transformers for Self-Supervised Real-World Denoising (NeurIPS 2024 · similarity 0.92)
- Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios (CVPR 2024 · similarity 0.90)
- APR-RD: Complemental Two Steps for Self-Supervised Real Image Denoising (AAAI 2025 · similarity 0.88)
- Attention-Only Transformers via Unrolled Subspace Denoising (ICML 2025 · similarity 0.87)
- Masked and Shuffled Blind Spot Denoising for Real-World Images (CVPR 2024 · similarity 0.85)
- Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.