Research map / Detection and segmentation / 3D detection and driving
SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SDAC, a multimodal CARLA-based synthetic dataset (cameras + LiDAR) that categorizes anomalies into object, scene, and scenario levels, and benchmarks closed-set, open-set, and anomaly segmentation methods on it.
Problem
Closed-set perception for autonomous driving fails on anomalies and corner cases (unknown objects, weather shifts, accidents), and existing anomaly datasets are mostly image-only and lack systematic anomaly levels and scenario-level corner cases.
Approach
A multimodal synthetic dataset generated in CARLA that organizes anomalies into object, scene, and scenario levels, with multi-view RGB, 64-channel LiDAR, 2D/3D boxes and segmentation masks, and parameterized pre-crash scenarios generated with Scenic.
Key results
- Closed-set 2D detectors drop on RainFog and Accident; e.g., Faster R-CNN mAP falls from 59.5 on Normal to 35.6 on RainFog and car/person AP on Accident-person is near 0.2.
- ORE fails to identify unknown objects (APu 0.04 on TruckBus and 0.0 on Props with extra data), while OpenDet reaches APu 55.5 on TruckBus but only 5.3 on Props.
- 3D detectors are less affected by weather: PV-RCNN AP_M 92.1 on Normal vs 92.0 on RainFog.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate image-based closed-set detectors on RainFog rather than Normal | Faster R-CNN, CenterNet, and D-DETR · SDAC-SNAP | mAP: 59.5 -> 35.6; 47.0 -> 29.3; 52.6 -> 33.0 | Hurt |
| Evaluate image-based closed-set detectors on Accident rather than Normal | Faster R-CNN, CenterNet, and D-DETR · SDAC-SNAP Accident | APp: 0.2; 0.2; 0.3 | Hurt |
| Evaluate LiDAR-based and multimodal detectors on RainFog rather than Normal | PointRCNN, PV-RCNN, and MVX-Net · SDAC-SNAP | APM: 81.8 -> 81.6; 92.1 -> 92.0; 90.1 -> 90.0 | No clear effect |
| Add vertical and diagonal flips and rotation (ablation) | Faster R-CNN · Accident-Car | AP: 32.4 -> 45.5 | Helped |
| Add vertical and diagonal flips and rotation (ablation) | Faster R-CNN · Accident-Person | AP: 0.2 -> 21.7 | Helped |
| Add vertical and diagonal flips and rotation (ablation) | CenterNet · Accident-Car | AP: 29.0 -> 51.2 | Helped |
| Add vertical and diagonal flips and rotation (ablation) | CenterNet · Accident-Person | AP: 0.2 -> 7.2 | Helped |
| Add vertical and diagonal flips and rotation (ablation) | D-DETR · Accident-Car | AP: 34.6 -> 36.2 | Helped |
| Add vertical and diagonal flips and rotation (ablation) | D-DETR · Accident-Person | AP: 0.3 -> 6.1 | Helped |
| Use ORE for unknown-object identification | ORE · TruckBus and Props | unknown-object AP50: 0.04; 0.02; 0.0 | Hurt |
| Use OpenDet for unknown-object identification | OpenDet · Props | unknown-object AP50: 5.3 | Hurt |
Surprising findings
- 3D LiDAR-based detectors are far less affected by weather changes than image-based detectors.
- Data augmentation (flips, rotation) helps only some models on the Accident subset; it does not fix the closed-set failure on lying pedestrians in general.
Limitations
- Synthetic CARLA data may have a domain gap to real-world sensor data.
- SDAC-SCENARIO has seven patterns and its scenario-level anomaly detection methods are not evaluated in the main text.
- 3D detection was not evaluated on the Accident subset due to flat-world assumptions.
Experimental setup
Datasets: SDAC-SNAP SDAC-SCENARIO COCO
Benchmarks: SDAC-SNAP Normal SDAC RainFog SDAC Accident SDAC TruckBus SDAC Props
Metrics: COCO mAP AP50 known/unknown Absolute Open-Set Error 3D AP40 (KITTI moderate) Cityscapes IoU mIoU AUROC FPR95
Baselines: Faster R-CNN CenterNet Deformable DETR PointRCNN PV-RCNN MVX-Net PSPNet DeepLabV3+ ORE OpenDet SML PEBAL
Builds on: CARLA Scenic CODA ORE OpenDet
Closest papers by meaning
- Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving (CVPR 2025 · similarity 0.89)
- The Road Ahead in Autonomous Driving: The KITScenes Multimodal Dataset (NeurIPS 2026 · similarity 0.86)
- When Every Millisecond Counts: Real-Time Anomaly Detection via the Multimodal Asynchronous Hybrid Network (ICML 2025 · similarity 0.85)
- Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes (AAAI 2024 · similarity 0.85)
- A Dataset for Semantic Segmentation in the Presence of Unknowns (CVPR 2025 · similarity 0.85)
- OpenAD: Open-World Autonomous Driving Benchmark for 3D Object Detection (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.