Research map / Image generation / Fast diffusion sampling
Effective Cloud Removal for Remote Sensing Images by an Improved Mean-Reverting Denoising Model with Elucidated Design Space
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EMRDM, a mean-reverting diffusion model for cloud removal in remote sensing images with an elucidated (EDM-style) design space, preconditioning, new samplers, and a temporal-fusion network for multi-temporal inputs.
Problem
Diffusion-based cloud removal typically starts from pure noise, ignoring the information in cloudy inputs and lacking pixel-level consistency; existing mean-reverting diffusion models are coupled and hard to modify.
Approach
Reformulates the mean-reverting forward SDE with simplified s(t)=1/(1+αt) and σ(t)=t, derives an ODE backward process, applies EDM-style preconditioning extended with the cloudy mean, and adds stochastic/deterministic samplers plus a temporal HDiT with temporal fusion self-attention for multi-temporal CR.
Key results
- On CUHK-CR1, EMRDM reaches 27.281 PSNR / 0.8007 SSIM / 0.0218 LPIPS, beating DE-MemoryNet (26.183 PSNR).
- On SEN12MS-CR, EMRDM achieves 32.14 PSNR and 0.924 SSIM, versus 31.77 PSNR for DiffCR.
- On Sen2 MTC New (L=3), EMRDM reaches 20.067 PSNR and 0.255 LPIPS, versus 19.150 PSNR for DiffCR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use EMRDM instead of the strongest baseline | Cloud removal models · CUHK-CR1 | PSNR: 26.183 -> 27.281 | Helped |
| Use EMRDM instead of the strongest baseline | Cloud removal models · CUHK-CR2 | PSNR: 24.348 -> 24.594 | Helped |
| Use EMRDM instead of the strongest baseline | Cloud removal models · SEN12MS-CR | PSNR: 31.77 -> 32.14 | Helped |
| Use EMRDM instead of the strongest baseline | Cloud removal models · Sen2 MTC New | PSNR: 19.150 -> 20.067 | Helped |
| Add cloudy images as conditioning inputs (ablation) | EMRDM · Sen2 MTC New | PSNR: 12.81 -> 18.26 | Helped |
| Add IR images as conditioning inputs (ablation) | EMRDM · Sen2 MTC New | PSNR: 18.26 -> 19.31 | Helped |
| Use the EMRDM mean-reverting framework (ablation) | EMRDM · Sen2 MTC New | PSNR: 19.31 -> 19.52 | Helped |
| Add preconditioning (ablation) | EMRDM · Sen2 MTC New | PSNR: 19.52 -> 19.47 | Hurt |
| Increase sequence length from L = 1 to L = 3 (ablation) | EMRDM multi-temporal network · Sen2 MTC New | PSNR: 16.09 -> 20.07 | Helped |
| Vary α across tested settings (ablation) | EMRDM · Sen2 MTC New | PSNR: 19.14 -> 20.00 | Mixed |
| Increase σmax across tested settings (ablation) | EMRDM · Sen2 MTC New | PSNR: 19.58 -> 20.03 | Mixed |
| Vary the number of sampling steps N (ablation) | EMRDM · Sen2 MTC New | PSNR: 20.00 -> 19.19 | Mixed |
Surprising findings
- Using a large number of sampling steps yields poor results; only five steps delivers superior performance, contrary to typical generative DM expectations.
Limitations
- Multi-temporal network requires a fixed sequence length L during training; the temporal benefit grows with L only on the evaluated datasets.
- Mono-temporal and multi-temporal results are evaluated on a limited set of four datasets.
Experimental setup
Datasets: CUHK-CR1 CUHK-CR2 SEN12MS-CR Sen2 MTC New
Benchmarks: CUHK-CR1 CUHK-CR2 SEN12MS-CR Sen2 MTC New
Metrics: PSNR SSIM LPIPS MAE SAM
Baselines: DiffCR DE-MemoryNet DE-MSDA MemoryNet UnCRtainTS DSen2-CR GLF-CR ACA-Net DDPM-CR CTGAN
Builds on: EDM IR-SDE HDiT UTAE L-TAE
Closest papers by meaning
- Residual Denoising Diffusion Models (CVPR 2024 · similarity 0.86)
- Building Bridges across Spatial and Temporal Resolutions: Reference-Based Super-Resolution via Change Priors and Conditional Diffusion Model (CVPR 2024 · similarity 0.85)
- Resfusion: Denoising Diffusion Probabilistic Models for Image Restoration Based on Prior Residual Noise (NeurIPS 2024 · similarity 0.84)
- AllClear: A Comprehensive Dataset and Benchmark for Cloud Removal in Satellite Imagery (NeurIPS 2024 · similarity 0.84)
- Diffusion Priors for Variational Likelihood Estimation and Image Denoising (NeurIPS 2024 · similarity 0.84)
- DiffusionSat: A Generative Foundation Model for Satellite Imagery (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.