Research map / Image generation / Image restoration
Rethinking Multi-Scale Representations in Deep Deraining Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MSDT, a multi-input multi-output Transformer for single image deraining that restores rain-free images coarse-to-fine across an image pyramid, using coupled representation modules (sparsity-compensated Transformer + frequency-enhanced conv) and gated fusion modules.
Problem
Existing Transformer-based image deraining methods rely on a fixed single-input single-output U-Net, ignoring explicit multi-scale input information and complementary implicit cross-scale information.
Approach
A multi-input multi-output Transformer that processes an image pyramid, restores rain-free outputs from coarsest to finest scale, uses neighbor-softmax sparse self-attention with per-scale thresholds, and fuses inter-scale features with gated fusion modules.
Key results
- Achieves 41.75 dB PSNR on Rain200L, exceeding DRSformer (41.23 dB) by 0.52 dB.
- Reaches 49.07 dB PSNR on SPA-Data, versus 48.54 dB for DRSformer.
- Ablation on Rain200H: single-scale S1 gives 32.08 dB PSNR, while all three scales give 32.45 dB.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MSDT instead of DRSformer | image deraining · Rain200L | PSNR: 41.23 -> 41.75 | Helped |
| Use MSDT instead of DRSformer | image deraining · Rain200H | PSNR: 32.17 -> 32.45 | Helped |
| Use MSDT instead of DRSformer | image deraining · DID-Data | PSNR: 35.35 -> 35.37 | Helped |
| Use MSDT instead of DRSformer | image deraining · DDN-Data | PSNR: 34.35 -> 34.36 | Helped |
| Use MSDT instead of DRSformer | image deraining · SPA-Data | PSNR: 48.54 -> 49.07 | Helped |
| Use MSDT instead of IDT | image deraining · SPA-Data | SSIM: 0.9930 -> 0.9926 | Hurt |
| Add scale S2 to S1 (ablation) | MSDT on Rain200H · Rain200H | PSNR: 32.08 -> 32.20 | Helped |
| Add scale S3 to S1+S2 (ablation) | MSDT on Rain200H · Rain200H | PSNR: 32.20 -> 32.45 | Helped |
| Add gated fusion module instead of direct feature concatenation (ablation) | MSDT on Rain200H · Rain200H | PSNR: 32.33 -> 32.45 | Helped |
| Add frequency-enhanced convolutional block (ablation) | MSDT on Rain200H · Rain200H | PSNR: 31.55 -> 32.45 | Helped |
| Use the full hybrid loss rather than MSC alone (ablation) | MSDT on Rain200H · Rain200H | PSNR: 32.17 -> 32.45 | Helped |
| Add MSFR loss (ablation) | MSDT on Rain200H · Rain200H | PSNR: 32.32 -> 32.45 | Helped |
Limitations
- FLOPs are higher than some Transformer baselines (e.g., 129.9G vs 61.9G for IDT and 45.9G for Uformer in Table 2).
- Per-scale thresholds for neighbor softmax are hand-set (0.6, 0.7, 0.8) rather than learned.
Experimental setup
Datasets: Rain200L Rain200H DID-Data DDN-Data SPA-Data
Benchmarks: Rain200L Rain200H DID-Data DDN-Data SPA-Data
Metrics: PSNR SSIM parameters FLOPs inference time
Baselines: DSC GMM DDN RESCAN PReNet MSPFN RCDNet MPRNet DualGCN SPDNet Uformer Restormer
Builds on: Restormer DRSformer MPRNet Multi-scale Charbonnier loss FFT-based frequency-enhanced block
Closest papers by meaning
- Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining (CVPR 2024 · similarity 0.91)
- Hybrid CNN-Transformer Feature Fusion for Single Image Deraining (AAAI 2023 · similarity 0.91)
- Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains (ICLR 2024 · similarity 0.89)
- FMRNet: Image Deraining via Frequency Mutual Revision (AAAI 2024 · similarity 0.87)
- Adapt or Perish: Adaptive Sparse Transformer with Attentive Feature Refinement for Image Restoration (CVPR 2024 · similarity 0.86)
- Channel Consistency Prior and Self-Reconstruction Strategy Based Unsupervised Image Deraining (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.