Research map / Image generation / Image restoration
Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Start
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds HFLS-Weather, a 1M-image physics-driven depth-based synthetic weather dataset, and trains weather-specific restorers with PIQO (GRPO-style parameter-perturbation RL using no-reference IQA rewards) plus a CLIP-driven multi-agent meta-controller for real-world adverse weather restoration.
Problem
Adverse weather image restoration models trained on synthetic data with fixed parameters generalize poorly to real-world degradations, and GRPO-style RL is hard to apply to restoration because outputs are deterministic and ground truth is unavailable.
Approach
Adapt GRPO to image restoration by injecting Gaussian parameter perturbations to generate diverse outputs, filtering by MUSIQ, and rewarding with a no-reference IQA ensemble; a global CLIP-based meta-controller then bids and sequences specialized restoration agents using historical success and IQA checks.
Key results
- On real snow, haze, and rain test sets the method reports Q-Align 3.9569 / 3.5608 / 4.0283 and CLIP-IQA 0.5918 / 0.4561 / 0.5623 (snow/haze/rain), highest among compared methods.
- Ablation on snow: Basic 0.4774 CLIP-IQA vs Basic+PIQO+Agent 0.5918; PIQO gives the larger share of the gain versus Agent alone.
- Multi-agent system latency is 570 ms versus OneRestore 17 ms, PromptIR 208 ms, DA-CLIP 6543 ms and JarvisIR 15250 ms.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Ours instead of the strongest baseline listed | Real-world restoration · Snow | CLIP-IQA: 0.5291 -> 0.5918 | Helped |
| Use Ours instead of the strongest baseline listed | Real-world restoration · Haze | CLIP-IQA: 0.3757 -> 0.4561 | Helped |
| Use Ours instead of the strongest baseline listed | Real-world restoration · Rain | CLIP-IQA: 0.4656 -> 0.5623 | Helped |
| Add Agent to Basic (ablation) | Basic restoration model · Snow | CLIP-IQA: 0.4774 -> 0.5242 | Helped |
| Add Agent to Basic (ablation) | Basic restoration model · Haze | CLIP-IQA: 0.3661 -> 0.3814 | Helped |
| Add Agent to Basic (ablation) | Basic restoration model · Rain | CLIP-IQA: 0.4392 -> 0.4721 | Helped |
| Add PIQO to Basic (ablation) | Basic restoration model · Snow | CLIP-IQA: 0.4774 -> 0.5653 | Helped |
| Add PIQO to Basic (ablation) | Basic restoration model · Haze | CLIP-IQA: 0.3661 -> 0.4496 | Helped |
| Add PIQO to Basic (ablation) | Basic restoration model · Rain | CLIP-IQA: 0.4392 -> 0.5340 | Helped |
| Add PIQO and Agent to Basic (ablation) | Basic restoration model · Snow | CLIP-IQA: 0.4774 -> 0.5918 | Helped |
| Add PIQO and Agent to Basic (ablation) | Basic restoration model · Haze | CLIP-IQA: 0.3661 -> 0.4561 | Helped |
| Add PIQO and Agent to Basic (ablation) | Basic restoration model · Rain | CLIP-IQA: 0.4392 -> 0.5623 | Helped |
Surprising findings
- The authors state that CLIP-IQA and Q-Align can be inflated and fail to capture subtle noise and residual artifacts on real images, motivating GPT-4o evaluation.
- Real-data cold start from RealSnow or Snow100K is weaker than HFLS-Weather cold start followed by PIQO, which the authors frame as evidence for high-quality synthetic cold starts.
Limitations
- Multi-agent system adds inference-time overhead from multi-round interactions.
- Real-world evaluation relies on no-reference IQA metrics and GPT-4o, which may be unreliable for subtle artifacts and lack ground truth.
Experimental setup
Datasets: HFLS-Weather SPA+ URHI WReal Snow100K RESIDE-OTS Google Landmark V2 OSV5M RTTS DDN-SIRR Real3000
Benchmarks: WReal real-world test subsets (snow, haze, rain)
Metrics: Q-Align CLIP-IQA LIQE MUSIQ GPT-4o artifact removal score GPT-4o weather resilience score GPT-4o overall visual quality score latency
Baselines: Chen et al. WGWS PromptIR OneRestore DA-CLIP DFPIR JarvisIR
Builds on: GRPO PPO DSANet CLIP Depth Anything V2
Closest papers by meaning
- Continuous Adverse Weather Removal via Degradation-Aware Distillation (CVPR 2025 · similarity 0.89)
- AWRaCLe: All-Weather Image Restoration Using Visual In-Context Learning (AAAI 2025 · similarity 0.87)
- Language-driven All-in-one Adverse Weather Removal (CVPR 2024 · similarity 0.87)
- Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal (CVPR 2024 · similarity 0.86)
- Multi-axis Prompt and Multi-dimension Fusion Network for All-in-one Weather-degraded Image Restoration (AAAI 2025 · similarity 0.86)
- MODEM: A Morton-Order Degradation Estimation Mechanism for Adverse Weather Image Recovery (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.