Research map / Label-efficient and robust learning / Domain adaptation and generalization
HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HPL-ESS, an unsupervised domain adaptation framework for event-based semantic segmentation that mixes self-training pseudo labels with pseudo labels from offline event-to-image reconstructions, refined by noisy label learning and a soft prototypical alignment module.
Problem
Event-based semantic segmentation lacks labeled event data; reconstruction-based pseudo labels are noisy and single-source pseudo labeling suffers from confirmation bias.
Approach
Combine self-training UDA (DaFormer-based teacher-student with source images) with a small (5%) set of offline E2VID reconstructions as a second pseudo-label source, refine these labels via noisy-label learning using teacher predictions, and align target event/reconstruction features to source class prototypes using a JS-divergence-based soft prototypical alignment.
Key results
- On DSEC-Semantic under UDA, HPL-ESS reaches 89.92% accuracy and 55.19% mIoU, versus 84.04% and 44.87% for ESS (+5.88% accuracy, +10.32% mIoU), and it also exceeds fully supervised ESS (89.37%, 53.29%).
- On DDD17 UDA, HPL-ESS achieves 88.65% accuracy and 53.51% mIoU, improving over ESS (87.86%, 52.46%).
- Using only 5% of event samples for reconstruction is optimal; the event count per sample drops from 2E6 to 1.8E5 (about 91% reduction).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HPL-ESS instead of ESS | Unsupervised event-based segmentation · DSEC-Semantic | accuracy: 84.04 -> 89.92 | Helped |
| use HPL-ESS instead of ESS | Unsupervised event-based segmentation · DSEC-Semantic | mIoU: 44.87 -> 55.19 | Helped |
| use HPL-ESS instead of ESS | Unsupervised event-based segmentation · DDD17 | accuracy: 87.86 -> 88.65 | Helped |
| use HPL-ESS instead of ESS | Unsupervised event-based segmentation · DDD17 | mIoU: 52.46 -> 53.51 | Helped |
| add ETI reconstructed images and hybrid pseudo labels (ablation) | Source-warmed DaFormer UDA baseline · DSEC-Semantic | mIoU: 44.87 -> 51.08 | Helped |
| add noisy label learning (NLL) (ablation) | Hybrid pseudo-labeling · DSEC-Semantic | mIoU: 51.08 -> 52.23 | Helped |
| add soft prototypical alignment (SPA) (ablation) | Hybrid pseudo-labeling · DSEC-Semantic | mIoU: 51.08 -> 52.69 | Helped |
| add both NLL and SPA (ablation) | Hybrid pseudo-labeling · DSEC-Semantic | mIoU: 51.08 -> 55.19 | Helped |
| increase reconstructed event sample proportion from 0% to 5% (ablation) | HPL-ESS · DSEC-Semantic | mIoU: 44.87 -> 55.19 | Helped |
| increase reconstruction proportion beyond 5% (ablation) | HPL-ESS · DSEC-Semantic | mIoU: 55.19 -> 55.15; 55.19 -> 54.96; 55.19 -> 54.82; 55.19 -> 54.51 | Hurt |
| repredict reconstruction pseudo labels during training instead of fixing them offline (ablation) | HPL-ESS · DSEC-Semantic | accuracy: 83.25 -> 89.92 | Helped |
| repredict reconstruction pseudo labels during training instead of fixing them offline (ablation) | HPL-ESS · DSEC-Semantic | mIoU: 48.45 -> 55.19 | Helped |
Surprising findings
- Using reconstructed images for all event samples hurts performance (mIoU drops to 54.51% at 100%), so only a small fraction is beneficial.
- Online re-prediction of reconstruction pseudo labels during training greatly outperforms fixed offline pseudo labels (55.19% vs 48.45% mIoU).
Limitations
- Performance on under-represented classes (e.g., rider, traffic light) remains lower due to class imbalance.
- Reconstructed images contain fuzzy regions and incorrect content, so the method still depends on noisy reconstruction-based labels.
- Evaluation limited to two driving datasets with CityScapes as the only source domain.
Experimental setup
Datasets: CityScapes DSEC-Semantic DDD17
Benchmarks: DSEC-Semantic DDD17
Metrics: accuracy mIoU
Baselines: ESS E2VID EV-Transfer VID2E EV-SegNet HALISE
Builds on: DaFormer ESS E2VID ClassMix EMA teacher
Closest papers by meaning
- Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach (AAAI 2025 · similarity 0.87)
- ESEG: Event-Based Segmentation Boosted by Explicit Edge-Semantic Guidance (AAAI 2025 · similarity 0.86)
- Towards Robust Pseudo-Label Learning in Semantic Segmentation: An Encoding Perspective (NeurIPS 2025 · similarity 0.86)
- Connectivity-Driven Pseudo-Labeling Makes Stronger Cross-Domain Segmenters (NeurIPS 2024 · similarity 0.86)
- OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies (CVPR 2024 · similarity 0.86)
- Label-Efficient Few-Shot Semantic Segmentation with Unsupervised Meta-Training (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.