Research map / Detection and segmentation / 3D detection and driving
Spotting the Unexpected (STU): A 3D LiDAR Dataset for Anomaly Segmentation in Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces STU, a real-world 3D LiDAR dataset with dense point-level anomaly and inlier labels plus surround-view cameras, and benchmarks adapted 2D anomaly segmentation methods on Mask4Former-3D.
Problem
Anomaly (road debris) segmentation for autonomous driving is well studied in 2D but lacks high-quality, publicly available 3D LiDAR datasets with dense labels and multimodal sensors.
Approach
A real-world LiDAR-camera dataset of staged and naturalistic road anomalies with dense point-level labels, designed so anomaly classes do not overlap with ID training classes, together with adaptations of 2D anomaly methods (MaxLogit, MC Dropout, Deep Ensembles, void classifier, RbA) to 3D Mask4Former.
Key results
- Deep Ensemble achieves the best object-level PQ of 10.43 among anomaly baselines and lowest FPR@95 of 58.05 (point-level), with AUROC 86.74.
- RbA reaches AUROC 66.38 with FPR@95 of 100.0, while MC Dropout reaches AUROC 61.51, showing weak transfer from 2D methods.
- Mask4Former-3D jointly trained on SemanticKITTI and STU-inlier reaches 52.7 PQ on STU-inlier vs 42.8 for SemanticKITTI-only training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Panoptic-CUDAL and STU-inlier training data (ablation) | Mask-PLS · STU-inlier validation | PQ: 39.6 -> 50.9 | Helped |
| Add Panoptic-CUDAL and STU-inlier training data (ablation) | Mask-PLS · SemanticKITTI validation | PQ: 59.9 -> 55.6 | Hurt |
| Add Panoptic-CUDAL and STU-inlier training data (ablation) | Mask4Former-3D · STU-inlier validation | PQ: 42.8 -> 52.7 | Helped |
| Add Panoptic-CUDAL and STU-inlier training data (ablation) | Mask4Former-3D · SemanticKITTI validation | PQ: 61.9 -> 60.7 | Hurt |
Surprising findings
- Anomalies are often predicted as inlier classes such as pedestrians or other vehicles with high confidence, so MaxLogit and ensembles fail to flag them.
- Directly transferring 2D anomaly segmentation methods to 3D LiDAR yields much weaker performance than in 2D settings.
Limitations
- Evaluation restricted to single-scan LiDAR; temporal sequence information not used in baselines.
- Only 70 anomaly sequences with objects placed in staged settings, with two naturalistic sequences.
- Evaluation limited to 50 m range and objects with at least five points.
Experimental setup
Datasets: SemanticKITTI Panoptic-CUDAL STU-inlier NuScenes
Benchmarks: STU SemanticKITTI STU-inlier
Metrics: AUROC FPR@95 AP PQ SQ RQ Recall Quality Unknown Quality
Baselines: Deep Ensemble MC Dropout Max Logit Void Classifier RbA Mask-PLS
Builds on: Mask4Former RbA MaxLogit Deep Ensembles MC Dropout
Closest papers by meaning
- Learning to Identify Out-of-Distribution Objects for 3D LiDAR Anomaly Segmentation (CVPR 2026 · similarity 0.92)
- Pixel-Wise Anomaly Detection in Complex Driving Scenes (CVPR 2021 · similarity 0.90)
- SDAC: A Multimodal Synthetic Dataset for Anomaly and Corner Case Detection in Autonomous Driving (AAAI 2024 · similarity 0.89)
- A Dataset for Semantic Segmentation in the Presence of Unknowns (CVPR 2025 · similarity 0.88)
- Self-Supervised Likelihood Estimation with Energy Guidance for Anomaly Segmentation in Urban Scenes (AAAI 2024 · similarity 0.87)
- NBA3D: Neighbor-Based Confidence Adjustment for 3D Rare Object Detection Using LiDAR (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.