Research map / Detection and segmentation / Open-set and few-shot detection
ERF: A Benchmark Dataset for Robust Semantic Segmentation Under Extreme Rainfall Conditions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the ERF dataset of real-world Singapore urban driving videos captured under violent rainfall (14,757 unlabeled and 100 labeled frames) and benchmarks foundation, domain generalization, domain adaptation, and video segmentation models on it.
Problem
Semantic segmentation models for autonomous driving are untested and fail under violent rainfall (>50 mm/h), a condition absent from existing rain datasets that mostly cover light rain.
Approach
A real-world, video-based urban scene dataset captured during four violent rainfall events in Singapore with depth/disparity data, plus a systematic benchmark of four method families showing none is robust to violent rain.
Key results
- Across 14 evaluated methods on ERF, the best mIoU is 62.4% (Rein on DINOv2), with no method robust to violent rain.
- Rein outperforms the second-best domain generalization method PASTA by 3.6 mIoU.
- HRDA in domain adaptation setting improves 2.4 mIoU over its domain generalization setting (50.0 vs 47.6).
- Video-based methods perform markedly worse, with best mIoU of 36.5% (SFC).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DINOv2 as the frozen encoder with a Mask2Former decoder instead of the other evaluated foundation-model encoders | Image-based foundation models · ERF | mIoU (%) | Helped |
| Use Rein rather than PASTA with DINOv2 | DINOv2-based domain generalization · ERF | mIoU (%): 3.6 mIoU (%) performance gain | Helped |
| Refine DINOv2 during training rather than using a frozen DINOv2 without refinement | DINOv2-based domain generalization · ERF | mIoU (%): 3.6 mIoU (%) performance gain | Helped |
| Add unlabeled target violent-rainfall images during training (ablation) | HRDA · ERF | mIoU (%): 2.4 mIoU (%) | Helped |
| Use video-based methods that incorporate temporal information instead of image-based methods | Semantic segmentation under violent rainfall · ERF | mIoU (%) | Hurt |
Surprising findings
- Video-based methods underperform image-based methods despite using temporal information.
- Domain adaptation methods are less effective than foundation-model-based domain generalization under violent rain.
Limitations
- Dataset collected in a single city (Singapore) with four violent-rain videos, limiting diversity.
- Only 100 labeled frames available for evaluation.
- Video-based methods use a lightweight Accel backbone, which may limit their performance.
Experimental setup
Datasets: Cityscapes Viper ERF
Benchmarks: ERF
Metrics: mIoU
Baselines: HRDA HGFormer PASTA Rein DAFormer MIC CDAC DA-VSN TPS SFC
Builds on: Cityscapes ACDC Rein PASTA HRDA
Closest papers by meaning
- Parsing All Adverse Scenes: Severity-Aware Semantic Segmentation with Mask-Enhanced Cross-Domain Consistency (AAAI 2024 · similarity 0.89)
- Semantic Segmentation in Multiple Adverse Weather Conditions with Domain Knowledge Retention (AAAI 2024 · similarity 0.88)
- End-to-End Video Semantic Segmentation in Adverse Weather using Fusion Blocks and Temporal-Spatial Teacher-Student Learning (NeurIPS 2024 · similarity 0.87)
- Semantic Segmentation on Raindrop Degraded Images Using Two-Stage Dual Teacher-Student Learning (AAAI 2025 · similarity 0.87)
- UniMix: Towards Domain Adaptive and Generalizable LiDAR Semantic Segmentation in Adverse Weather (CVPR 2024 · similarity 0.86)
- Towards Universal Rainy Image Restoration: Benchmark and Baseline (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.