Research map / Vision-language and multimodal / Human and embodied modeling

Language-driven All-in-one Adverse Weather Removal

CVPR 2024 · Vision-language and multimodal › Human and embodied modeling · approach: Medical vision-language models · 36 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LDR, an all-in-one adverse weather removal framework that uses a pre-trained vision-language model (LISA) to generate degradation descriptions and a sparse top-K mixture-of-experts of convolution filters to restore images without weather-type labels.

Problem

All-in-one adverse weather restoration typically needs auxiliary supervision such as weather type or depth labels and uses fixed sub-network structures, which limits the diversity of weather-specific knowledge and generalization to unseen or mixed weather.

Approach

Query a pre-trained vision-language model with a chain-of-thought-style question about occurrence, type and severity of degradation, convert the textual description into a pixel-wise degradation map via cross-attention, and use it to pixel-wise select top-K convolution experts from a candidate list (sparse MoE) and aggregate features, avoiding ground-truth weather labels.

Key results

• On the All-weather dataset, LDR achieves average PSNR/SSIM of 29.75/0.916, above the best baseline WGWS at 28.78/0.909.
• On WeatherStream, LDR reaches 23.55/0.822 average PSNR/SSIM, versus 23.10/0.813 for WGWS.
• Ablation: using all three modules (degradation map measurement, top-K expert restoration, restoration feature aggregation) gives 29.75 dB PSNR versus 27.93 dB with only top-K expert restoration.

Measured effects reported in the paper

Surprising findings

• Zero-outing computations of unrelated parameters in Transweather barely changes restoration quality, motivating sparse expert selection.
• The BestT + VL baseline (zero-shot PVL classification plus best task-specific model) performs far below the BestT + GT oracle, indicating classification accuracy matters.

Limitations

• Relies on auxiliary inference from a large pre-trained vision-language model, adding cost not quantified in the paper.
• Evaluated on only three weather types in the main benchmarks, with haze and mixed weather shown only qualitatively.

Experimental setup

Datasets: All-weather dataset WeatherStream

Benchmarks: All-weather dataset WeatherStream

Metrics: PSNR SSIM

Baselines: MPRNet NAFNet Uformer Restormer GRL All-in-One AirNet TUM Transweather WDiff WGWS

Builds on: AirNet TUM Restormer LISA

Closest papers by meaning

• Continuous Adverse Weather Removal via Degradation-Aware Distillation (CVPR 2025 · similarity 0.88)
• Semantic Segmentation in Multiple Adverse Weather Conditions with Domain Knowledge Retention (AAAI 2024 · similarity 0.87)
• AWRaCLe: All-Weather Image Restoration Using Visual In-Context Learning (AAAI 2025 · similarity 0.87)
• Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Start (NeurIPS 2025 · similarity 0.87)
• WeatherGFM: Learning a Weather Generalist Foundation Model via In-context Learning (ICLR 2025 · similarity 0.86)
• Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal (CVPR 2024 · similarity 0.85)