Research map / Video / Video segmentation and tracking
Genuine Knowledge from Practice: Diffusion Test-Time Adaptation for Video Adverse Weather Removal
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diff-TTA, a diffusion-based video adverse weather removal network with a temporal noise model and test-time adaptation via a tubelet self-calibration proxy task inside the diffusion reverse process.
Problem
Video adverse weather removal models trained on seen weather fail to generalize to unseen weather conditions at test time, and existing test-time adaptation is costly and poorly suited to low-level restoration.
Approach
Replaces i.i.d. diffusion noise with an ARMA-style temporally correlated noise model for training a NAFNet-based denoiser with 3D conv first/last layers, and performs online test-time adaptation by optimizing a noise-prediction proxy loss on cropped tubelets at each denoising step of the DDIM reverse process.
Key results
- Average PSNR/SSIM on seen weather (rain, haze, snow) of 29.63 / 0.9453, versus 29.17 / 0.9394 for ViWS-Net.
- On unseen VRDS, Diff-TTA reaches 22.57 PSNR / 0.7281 SSIM, above ViWS-Net's 21.57 / 0.7094.
- Diff-TTA runs in 6.01 s per 5-frame clip versus 542.76 s for WeatherDiffusion, stated as about 90x more efficient.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the diffusion process to vanilla NAFNet (ablation) | Diff-TTA framework · Average (Rain, Haze, Snow) | PSNR: 26.83 -> 28.52 | Helped |
| add the diffusion process to vanilla NAFNet (ablation) | Diff-TTA framework · Average (Rain, Haze, Snow) | SSIM: 0.9264 -> 0.9342 | Helped |
| replace vanilla noise with the temporal noise model (ablation) | diffusion model M2 · Average (Rain, Haze, Snow) | PSNR: 28.52 -> 29.15 | Helped |
| replace vanilla noise with the temporal noise model (ablation) | diffusion model M2 · Average (Rain, Haze, Snow) | SSIM: 0.9342 -> 0.9425 | Helped |
| add Diff-TTA (ablation) | M3 · Average (Rain, Haze, Snow) | PSNR: 29.15 -> 29.63 | Helped |
| add Diff-TTA (ablation) | M3 · Average (Rain, Haze, Snow) | SSIM: 0.9425 -> 0.9453 | Helped |
| use Diff-TTA instead of WeatherDiffusion | video weather removal inference · 5-frame video clip, 256×256 patch | Run-time (s): 542.76 -> 6.01 | Helped |
Surprising findings
- Ablation shows the temporal noise model adds 0.63 dB average PSNR over the standard diffusion process, and test-time adaptation adds 0.48 dB more.
Limitations
- Inference is still slower than feed-forward video restoration methods; the authors plan to accelerate it for real-time use.
- Unseen-weather evaluation uses only 8 randomly sampled 20-frame videos per dataset.
Experimental setup
Datasets: RainMotion REVIDE KITTI-snow VRDS RVSD
Benchmarks: RainMotion REVIDE KITTI-snow VRDS RVSD
Metrics: PSNR SSIM run-time FLOPs
Baselines: ViWS-Net WeatherDiffusion TransWeather TKL All-in-one UVRNet WGWS-Net PReNet RDD-Net EDVR
Builds on: DDIM DDPM NAFNet ARMA SHOT
Closest papers by meaning
- End-to-End Video Semantic Segmentation in Adverse Weather using Fusion Blocks and Temporal-Spatial Teacher-Student Learning (NeurIPS 2024 · similarity 0.88)
- Continuous Adverse Weather Removal via Degradation-Aware Distillation (CVPR 2025 · similarity 0.87)
- Real-World Adverse Weather Image Restoration via Dual-Level Reinforcement Learning with High-Quality Cold Start (NeurIPS 2025 · similarity 0.86)
- NightRain: Nighttime Video Deraining via Adaptive-Rain-Removal and Adaptive-Correction (AAAI 2024 · similarity 0.86)
- Semi-Supervised State-Space Model with Dynamic Stacking Filter for Real-World Video Deraining (CVPR 2025 · similarity 0.85)
- Language-driven All-in-one Adverse Weather Removal (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.