Research map / Detection and segmentation / CNN features
ESEG: Event-Based Segmentation Boosted by Explicit Edge-Semantic Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ESEG, an event-based semantic segmentation framework that adds explicit semantic-edge supervision (labels generated via a SAM-based pipeline, SELSAM) and a D2CAF fusion module using density indicators, cross-attention, and dynamic window masking.
Problem
Event-based semantic segmentation struggles because event data are noisy and sparse, so models have difficulty locating and extracting reliable cues for pixel-level prediction; no existing event segmentation datasets provide semantic edge labels.
Approach
Instead of expecting the network to implicitly find reliable edge regions in sparse event data, explicitly supervise an edge-semantic branch (DFF/ResNet34) with SAM-derived semantic-edge labels and use its features to guide a SegFormer dense-semantic branch through a density-aware, edge-masked cross-attention fusion module (D2CAF).
Key results
- On DSEC-Semantic, ESEG-L reaches 57.55 mIoU and 91.47 Acc, versus 54.19 mIoU for the SegFormer baseline (ESEG-B 55.93 mIoU).
- On DDD17, ESEG-L reaches 59.97 mIoU, outperforming EvDistill (58.02) and ESS-Sup (56.67).
- Ablation on DSEC with ESEG-B: moving from plain concatenation fusion (54.48 mIoU) to full D2CAF (55.93 mIoU) improves accuracy; removing edge supervision (no fusion) gives 54.19.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add explicit edge-semantic supervision and D2CAF to SegFormer | ESEG-B · DSEC | mIoU: 54.19 -> 55.93 | Helped |
| Add explicit edge-semantic supervision and D2CAF to SegFormer | ESEG-B · DDD17 | mIoU: 52.76 -> 57.01 | Helped |
| Add concatenation fusion instead of no fusion (ablation) | ESEG-B · DSEC | mIoU: 54.19 -> 54.48 | Helped |
| Replace concatenation with cross-attention fusion (ablation) | ESEG-B · DSEC | mIoU: 54.48 -> 54.83 | Helped |
| Add DIM mapping to cross-attention fusion (ablation) | ESEG-B · DSEC | mIoU: 54.83 -> 55.22 | Helped |
| Add basic aggregation window masking (ablation) | ESEG-B · DSEC | mIoU: 55.22 -> 55.33 | Helped |
| Replace basic window mask with dynamic window masking (ablation) | ESEG-B · DSEC | mIoU: 55.33 -> 55.93 | Helped |
Surprising findings
- ESEG-B is slightly below EvDistill on DDD17 mIoU, which the authors attribute to poor DDD17 image quality and insufficient training.
Limitations
- Results on DDD17 for the base model trail EvDistill, attributed to lower image quality and training difficulty.
- Edge-semantic labels depend on SAM outputs, and the paper does not evaluate label quality beyond visualization.
Experimental setup
Datasets: DSEC-Semantic DDD17
Benchmarks: DSEC-Semantic DDD17
Metrics: mIoU Accuracy
Baselines: SegFormer Ev-SegNet EvSegFormer SpikingEDN HMNet-L ESS-Sup ESS EvDistill Vid2E E2VID
Builds on: SegFormer DFF SAM Ev-SegNet PiDiNet
Closest papers by meaning
- Efficient Event-Based Semantic Segmentation via Exploiting Frame-Event Fusion: A Hybrid Neural Network Approach (AAAI 2025 · similarity 0.90)
- Re-coding for Uncertainties: Edge-awareness Semantic Concordance for Resilient Event-RGB Segmentation (NeurIPS 2025 · similarity 0.89)
- OpenESS: Event-based Semantic Scene Understanding with Open Vocabularies (CVPR 2024 · similarity 0.88)
- EventPillars: Pillar-based Efficient Representations for Event Data (AAAI 2025 · similarity 0.88)
- LaSe-E2V: Towards Language-guided Semantic-aware Event-to-Video Reconstruction (NeurIPS 2024 · similarity 0.87)
- HPL-ESS: Hybrid Pseudo-Labeling for Unsupervised Event-based Semantic Segmentation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.