Research map / Vision-language and multimodal / Multimodal LLMs
SnowMaster: Comprehensive Real-world Image Desnowing via MLLM with Multi-Model Feedback Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RealSnow10K, a 10K real snowy image dataset, and MMPO (multi-model DPO on 36K expert-ranked pairs) to fine-tune Q-Instruct as a snow-aware evaluator, which drives mean-teacher semi-supervised desnowing with NAFNet.
Problem
Real-world image desnowing generalizes poorly because models train on synthetic snow, and real snowy images lack reference ground truth and snow-specific evaluation metrics to be used for training.
Approach
Extends DPO to multi-model preference pairs (outputs of several MLLMs ranked by experts) to fine-tune Q-Instruct on snow-specific perception, then uses its logit-weighted multiple-choice score as a pseudo-label filter in a mean-teacher semi-supervised desnowing framework.
Key results
- MMPO raised Q-Instruct-13b accuracy on snowflake amount from 76.31 to 82.14 and on obscure from 73.26 to 80.90.
- SnowMaster achieved BRISQUE 16.305 on Snow100K-realistic versus 17.566 for NAFNet, and 17.546 on RealSnow10K versus 18.564 for TransWeather.
- Ablation: multi-angle evaluation reduced BRISQUE from 18.21 (single-angle) to 17.55.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add semi-supervised learning (ablation) | SnowMaster · RealSnow10K | BRISQUE: 22.52 -> 17.55 | Helped |
| add semi-supervised learning (ablation) | SnowMaster · RealSnow10K | NIQE: 3.33 -> 3.06 | Helped |
| add the MLLM evaluator (ablation) | SnowMaster · RealSnow10K | BRISQUE: 20.36 -> 17.55 | Helped |
| add the MLLM evaluator (ablation) | SnowMaster · RealSnow10K | NIQE: 3.25 -> 3.06 | Helped |
| add MMPO alignment (ablation) | SnowMaster · RealSnow10K | BRISQUE: 18.98 -> 17.55 | Helped |
| add MMPO alignment (ablation) | SnowMaster · RealSnow10K | NIQE: 3.19 -> 3.06 | Helped |
| use multi-angle instead of single-angle evaluation (ablation) | SnowMaster · RealSnow10K | BRISQUE: 18.21 -> 17.55 | Helped |
| use multi-angle instead of single-angle evaluation (ablation) | SnowMaster · RealSnow10K | NIQE: 3.12 -> 3.06 | Helped |
| apply MMPO fine-tuning | Q-Instruct-13B · Snowflake amount evaluation | accuracy: 76.31 -> 82.14 | Helped |
| apply MMPO fine-tuning | Q-Instruct-13B · Object obscure evaluation | accuracy: 73.26 -> 80.90 | Helped |
| apply MMPO fine-tuning | Q-Instruct-13B · Texture evaluation | accuracy: 58.36 -> 64.95 | Helped |
| apply MMPO fine-tuning | Q-Instruct-13B · Background clarity evaluation | accuracy: 63.42 -> 68.58 | Helped |
Surprising findings
- MMPO-tuned Q-Instruct does not surpass LLaVA-1.6-13b on the snowflake amount indicator, which the authors attribute to training on outputs of those same models.
- Texture and background accuracy improved even though only snowflake amount and obscure were targeted for optimization.
Limitations
- Computational intensity from running multiple MLLM evaluations.
- Preference data covers only two indicators and relies on a small set of expert rankings (1,500 images).
Experimental setup
Datasets: RealSnow10K Snow100K
Benchmarks: Snow100K-realistic RealSnow10K
Metrics: BRISQUE NIQE PIQE PAQ2-PIQ accuracy on snowflake amount/obscure/texture/background
Baselines: SMGARN NAFNet TransWeather Snowformer T3-DiffWeather Q-Instruct-13b LLaVA-1.5-13b LLaVA-1.6-13b MiniCPM-V
Builds on: DPO Q-Instruct Q-Align mean teacher framework
Closest papers by meaning
- Learning Real-World Image De-weathering with Imperfect Supervision (AAAI 2024 · similarity 0.85)
- Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Start (NeurIPS 2025 · similarity 0.84)
- VLM-RobustBench: A Comprehensive Benchmark for Robustness of Vision-Language Models (ICML 2026 · similarity 0.84)
- Benchmarking Large Vision-Language Models on Fine-Grained Image Tasks: A Comprehensive Evaluation (ICLR 2026 · similarity 0.83)
- Continuous Adverse Weather Removal via Degradation-Aware Distillation (CVPR 2025 · similarity 0.83)
- iLLaVA: An Image is Worth Fewer Than 1/3 Input Tokens in Large Multimodal Models (ICLR 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.