Research map / Image generation / Image restoration
Pseudo-Siamese Blind-spot Transformers for Self-Supervised Real-World Denoising
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SelfFormer, a transformer-based self-supervised real-world denoiser that builds a blind-spot structure into attention via directional half-plane grid self-attention and trains a pseudo-Siamese pair (directional and full-grid) with mutual learning.
Problem
Self-supervised real-world image denoising from single noisy images, where blind-spot networks struggle because real noise is spatially correlated and transformers have not been well integrated with blind-spot constraints.
Approach
Directional self-attention (DSA) restricts attention to a half-plane grid excluding the token itself, giving a built-in blind-spot without input masking; a pseudo-Siamese pair of SelfFormer-D (directional, four rotated branches with shared weights) and SelfFormer-F (full grid SA, used for inference) is trained with a mutual L1 loss so the full-grid network regularizes and benefits from the directi
Key results
- SelfFormer reaches 37.69 dB / 0.937 SSIM on SIDD benchmark and 38.92 dB / 0.943 on DND, the best among clean-image-free methods.
- On NIND ISO3200 and ISO5000 it scores 34.43 and 33.55 dB, above PUCA (34.24, 33.49).
- Ablation: replacing DSA with grid SA drops SIDD-Validation PSNR to 23.96 dB, showing the blind-spot property is essential.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SelfFormer instead of the strongest listed clean-image-free baseline | SelfFormer · SIDD-Benchmark | PSNR (dB): 37.54 -> 37.69 | Helped |
| Use SelfFormer instead of the strongest listed clean-image-free baseline | SelfFormer · DND benchmark | PSNR (dB): 38.83 -> 38.92 | Helped |
| Use SelfFormer instead of the strongest listed clean-image-free baseline | SelfFormer · NIND ISO3200 | PSNR (dB): 34.41 -> 34.43 | Helped |
| Use SelfFormer instead of the strongest listed clean-image-free baseline | SelfFormer · NIND ISO5000 | PSNR (dB): 33.52 -> 33.55 | Helped |
| Add directional self-attention and the four-branch structure instead of grid SA with one branch (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 23.96 -> 37.63 | Helped |
| Add channel attention (CA) modules instead of replacing them with DSA modules (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.54 -> 37.63 | Helped |
| Add mutual learning and the Siamese structure (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.29 -> 37.63 | Helped |
| Add a blind-spot mechanism and mutual learning to SelfFormer-F (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 23.66 -> 37.63 | Helped |
| Add DSA modules instead of using BSCA modules in their place (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.22 -> 37.63 | Helped |
| Share weights between SelfFormer-D and SelfFormer-F (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.45 -> 37.63 | Helped |
| Increase grid size from 8 to 20 (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.43 -> 37.65 | Helped |
| Set loss weight λ too small or too large rather than the reported setting 1 (ablation) | SelfFormer · SIDD-Validation | PSNR (dB): 37.63 -> 37.35; 37.63 -> 37.51 | Hurt |
Surprising findings
- Using only SelfFormer-F (full grid, no blind-spot) without mutual learning collapses to 23.66 dB, indicating overfitting to the noisy image.
Limitations
- Transformer-based models incur higher computational cost than CNN-based models.
- Significant performance gap remains between SelfFormer and supervised denoisers.
- Training memory is high due to the four-branch SelfFormer-D.
Experimental setup
Datasets: SIDD-Medium SIDD-Validation
Benchmarks: SIDD-Benchmark DND NIND
Metrics: PSNR SSIM
Baselines: AP-BSN LG-BPN PUCA SS-BSN MM-BSN C-BSN SASL Laine et al.
Builds on: Laine19 directional convolution blind-spot Noise2Void grid self-attention NAFNet PUCA
Closest papers by meaning
- Rethinking Transformer-Based Blind-Spot Network for Self-Supervised Image Denoising (AAAI 2025 · similarity 0.92)
- Exploring Efficient Asymmetric Blind-Spots for Self-Supervised Denoising in Real-World Scenarios (CVPR 2024 · similarity 0.90)
- APR-RD: Complemental Two Steps for Self-Supervised Real Image Denoising (AAAI 2025 · similarity 0.89)
- Masked and Shuffled Blind Spot Denoising for Real-World Images (CVPR 2024 · similarity 0.88)
- Zero-Shot Blind-spot Image Denoising via Implicit Neural Sampling (CVPR 2025 · similarity 0.86)
- Positive2Negative: Breaking the Information-Lossy Barrier in Self-Supervised Single Image Denoising (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.