Research map / Image generation / Image restoration
Harnessing Joint Rain-/Detail-aware Representations to Eliminate Intricate Rains
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoIC, which pairs a joint rain-/detail-aware contrastive learning strategy with a context-based instance-level modulation mechanism to train CNN and Transformer deraining models on mixed datasets.
Problem
Training a single image deraining model on mixed rain datasets overlooks discrepancies in rain density, type, and background, leading to suboptimal optimization and poor real-world deraining.
Approach
Learn an instance-level embedding that encodes both rain and background detail via contrastive learning with rain-aware (swapped rain layer) and detail-aware (blurred clean image) negatives, then use this embedding to modulate CNN convolution weights (via adaptive softmax-based weight generation) and Transformer keys/values (via cross-attention context) layer-wise.
Key results
- CoIC improves BRN, RCDNet, DGUNet, IDT, and DRSformer on mixed synthetic datasets; e.g., BRN Rain800 PSNR rises from 24.15 to 26.13 dB.
- DRSformer + CoIC reaches 32.26 dB PSNR on Rain200H when trained individually, the best among compared methods there (vs 32.17 for DRSformer).
- Adding SPAData to two-stage training, DRSformer + CoIC reaches 46.03 dB on SPAData vs 45.46 dB without CoIC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CoIC to DRSformer | DRSformer trained on mixed synthetic datasets · Rain800 | PSNR: 29.86 -> 29.92 | Helped |
| Add CoIC to DRSformer | DRSformer trained on mixed synthetic datasets · Rain200L | PSNR: 39.74 -> 39.81 | Helped |
| Add CoIC to DGUNet | DGUNet trained on mixed synthetic datasets · Rain200H | PSNR: 31.00 -> 31.07 | Helped |
| Add CoIC to DGUNet | DGUNet evaluated on real rainy images · RealInt | NIQE: 4.7040 -> 4.6008 | Helped |
| Add CoI-M without contrastive loss (ablation) | DGUNet trained on Rain200L, Rain200H, and Rain800 · Rain800 | PSNR: 31.10 -> 31.16 | Helped |
| Add rain-aware negative exemplars alone (ablation) | DGUNet with CoI-M and contrastive loss · Rain800 | PSNR: 31.16 -> 31.10 | No clear effect |
| Add detail-aware negative exemplars (ablation) | DGUNet with CoI-M and contrastive loss · Rain800 | PSNR: 31.16 -> 31.20 | Helped |
| Add both rain- and detail-aware negative exemplars (ablation) | DGUNet with CoI-M and contrastive loss · Rain800 | PSNR: 31.10 -> 31.33 | Helped |
| Add contrastive loss (ablation) | DGUNet with CoI-M · Rain200H | PSNR: 31.46 -> 31.48 | Helped |
| Train DRSformer with CoIC using 128×128 instead of 96×96 patches (ablation) | DRSformer + CoIC · Rain800 | PSNR: 29.92 -> 30.21 | Helped |
| Add CoIC during two-stage training with SPAData | DRSformer · SPAData | PSNR: 45.46 -> 46.03 | Helped |
| Add CoIC when training from scratch on synthetic and SPAData | DRSformer · SPAData | PSNR: 46.52 -> 46.76 | Helped |
Surprising findings
- Rain-aware negative exemplars alone gave no improvement on Rain800, attributed to the encoder finding a trivial solution.
- Training DRSformer from scratch on synthetic plus SPAData gave higher SPAData PSNR (46.52 dB without CoIC) than two-stage training, while the CoIC gains on synthetic sets shrank.
Limitations
- Gains on some datasets are marginal (e.g., DRSformer on Rain200L and Rain800 with patch size 128 shows minimal or negative change).
- Gaussian blur may not model degradation in excessively rainy images with heavy occlusion, as the authors note.
- Increased parameters and testing time from CoIC, especially for DRSformer (+14.12M params).
Experimental setup
Datasets: Rain200L Rain200H Rain800 DID-Data DDN-Data SPAData RealInt
Benchmarks: Rain200L Rain200H Rain800 DID-Data DDN-Data RealInt SPAData
Metrics: PSNR SSIM NIQE
Baselines: Syn2Real DCD-GAN Rain13K-trained models DDN DID-MDN JORDER-E PReNet RCDNet DGCN SPDNet Restormer Uformer
Builds on: MoCo Syn2Real LoRA
Closest papers by meaning
- Bidirectional Multi-Scale Implicit Neural Representations for Image Deraining (CVPR 2024 · similarity 0.91)
- From Rain Generation to Rain Removal (CVPR 2021 · similarity 0.90)
- Boosting Image De-Raining via Central-Surrounding Synergistic Convolution (AAAI 2025 · similarity 0.90)
- Rethinking Multi-Scale Representations in Deep Deraining Transformer (AAAI 2024 · similarity 0.89)
- Towards Universal Rainy Image Restoration: Benchmark and Baseline (AAAI 2025 · similarity 0.88)
- Semantic Segmentation on Raindrop Degraded Images Using Two-Stage Dual Teacher-Student Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.