Research map / Detection and segmentation / Open-set and few-shot detection
Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SLEEG, a self-supervised anomaly segmentation framework that trains a frozen-segmentation model's extra OoD head with copy-paste pseudo anomalies refined by energy-based likelihood estimators, requiring no auxiliary anomaly data.
Problem
Anomaly segmentation for autonomous driving: designing a good anomaly score for pixels and generating training anomaly samples without costly labeled anomaly data or retraining segmentation networks.
Approach
Treats segmentation decoders as energy functions and adds a learnable OoD head trained by maximizing anomaly likelihood of self-generated copy-paste anomaly patches; combines a task-agnostic binary estimator with a task-oriented residual of joint energy (JEM), and refines pasted masks via likelihood-guided variance-minimizing thresholding.
Key results
- On the Fishyscapes LAF test set, SLEEG reaches 59.66 AP (best in its group) with 6.69 FPR95 without auxiliary OoD data or re-training.
- On Road Anomaly test, SLEEG achieves 42.60 FPR95 and 38.10 AP, outperforming most non-retraining methods such as SML (25.82 AP).
- On FS LAF validation, SLEEG reaches 70.90 AP and 10.90 FPR95, a large gain over GMMSeg (43.47 AP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TAE to the JEM baseline (ablation) | SLEEG · FS LAF validation | AP: 31.0 -> 57.5 | Helped |
| Add TORE to the JEM baseline (ablation) | SLEEG · FS LAF validation | AP: 31.0 -> 47.9 | Helped |
| Combine TAE and TORE (ablation) | SLEEG · FS LAF validation | AP: 31.0 -> 70.9 | Helped |
| Combine TAE and TORE (ablation) | SLEEG · Road Anomaly test | AP: 18.9 -> 38.1 | Helped |
| Replace static γ with dynamic γ̂ (ablation) | TORE · FS LAF validation | AP: 65.2 -> 70.9 | Helped |
| Replace static γ with dynamic γ̂ (ablation) | TORE · Road Anomaly test | AP: 34.4 -> 38.1 | Helped |
| Add mask refinement to square patches (ablation) | SLEEG · FS LAF validation | AP: 45.7 -> 62.5 | Helped |
| Add mask refinement to square patches (ablation) | SLEEG · Road Anomaly test | AP: 18.2 -> 34.8 | Helped |
| Use convex patches instead of square patches without refinement (ablation) | SLEEG · FS LAF validation | AP: 45.7 -> 61.9 | Helped |
| Add mask refinement to convex patches (ablation) | SLEEG · FS LAF validation | AP: 61.9 -> 70.9 | Helped |
| Add mask refinement to convex patches (ablation) | SLEEG · Road Anomaly test | AP: 35.9 -> 38.1 | Helped |
| Use square patches instead of convex patches without refinement (ablation) | SLEEG · FS LAF validation | AP: 61.9 -> 45.7 | Hurt |
Surprising findings
- Simple square patches with refinement outperform naive void-class outlier exposure on some metrics, and refinement helps even for square patches.
- SLEEG underperforms methods using auxiliary OoD data on the FS Static benchmark, which the authors attribute to overlap between auxiliary data and synthetic anomalies.
Limitations
- Inferior to methods using auxiliary OoD data on FS Static.
- Relies on a segmentation model pretrained on Cityscapes; generalization gap to Road Anomaly domain shift.
- Anomaly likelihood is unstable early in training, requiring a JEM warmup for mask refinement.
Experimental setup
Datasets: Cityscapes Fishyscapes Lost & Found Fishyscapes Static
Benchmarks: Fishyscapes Lost & Found Fishyscapes Static Road Anomaly Lost & Found SMIYC StreetHazards
Metrics: FPR95 AP AUROC
Baselines: SML JEM Void Class DenseHybrid PEBAL CoroCL GMMSeg FED-C FED-U SynthCP Synboost MSP
Builds on: joint energy-based model (JEM) PEBAL SML
Closest papers by meaning
- Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving (CVPR 2025 · similarity 0.87)
- FAST: Foreground‑aware Diffusion with Accelerated Sampling Trajectory for Segmentation‑oriented Anomaly Synthesis (NeurIPS 2025 · similarity 0.85)
- SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving (AAAI 2024 · similarity 0.85)
- A Dataset for Semantic Segmentation in the Presence of Unknowns (CVPR 2025 · similarity 0.85)
- Promptable Anomaly Segmentation with SAM Through Self-Perception Tuning (AAAI 2025 · similarity 0.85)
- PANDA: Towards Generalist Video Anomaly Detection via Agentic AI Engineer (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.