Research map / Detection and segmentation / Open-set and few-shot detection

Segment Every Out-of-Distribution Object

CVPR 2024 · Detection and segmentation › Open-set and few-shot detection · approach: Amodal and open-world instance segmentation · 14 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes S2M, which converts anomaly score maps from existing OoD detectors into box prompts via a trained Faster R-CNN prompt generator and feeds them to SAM to segment whole OoD objects without threshold selection.

Problem

Anomaly-score-based OoD detection in semantic segmentation yields fragmented, inaccurate masks for OoD objects, and the required threshold is narrow and hard to pick without validation data.

Approach

Instead of thresholding per-pixel anomaly scores, convert the anomaly score map into box prompts with a detector trained on outlier-exposure synthetic data, then segment each box with SAM and take the union of masks.

Key results

• S2M achieves IoU of 69.99 on FS Static and 77.54 on SMIYC-Anomaly vs. best baseline RPL+CoroCL at 68.80 on FS Static and 50.97 on RoadAnomaly region, with ~20% average IoU gain claimed.
• On SMIYC-Obstacle, S2M outperforms RPL by 38.98% in IoU (67.64 vs 28.66).
• S2M reduces FPR95 on SMIYC Anomaly to 1.04 vs 7.18 for RPL+CoroCL, and inference adds only about 0.059s per image over RPL (0.217s to 0.276s).

Measured effects reported in the paper

Surprising findings

• Using PEBAL anomaly scores in S2M also improves IoU substantially over PEBAL alone (e.g., 8.1 to 40.8 on SMIYC-Obstacle IoU is reported as 6.74 to 40.77).

Limitations

• Fails when OoD objects are very small or translucent, so the anomaly map contains no usable location cue.
• Depends on the quality of the upstream anomaly score; the prompt generator cannot produce boxes if the map is uninformative.
• Relies on synthetic outlier-exposure training data rather than real OoD examples.

Experimental setup

Datasets: Cityscapes COCO Fishyscapes SMIYC RoadAnomaly

Benchmarks: Fishyscapes Static Fishyscapes Lost & Found SMIYC Anomaly SMIYC Obstacle RoadAnomaly

Metrics: IoU AuIoU mean F1 AuPRC FPR95

Baselines: RPL+CoroCL PEBAL Synboost DenseHybrid Meta-OoD Maximum softmax Mahalanobis SML

Builds on: RPL PEBAL Segment Anything Faster R-CNN Outlier Exposure

Closest papers by meaning

• Extremely Simple Multimodal Outlier Synthesis for Out-of-Distribution Detection and Segmentation (NeurIPS 2025 · similarity 0.84)
• A Dataset for Semantic Segmentation in the Presence of Unknowns (CVPR 2025 · similarity 0.84)
• Conjugated Semantic Pool Improves OOD Detection with Pre-trained Vision-Language Models (NeurIPS 2024 · similarity 0.84)
• Image Background Serves as Good Proxy for Out-of-distribution Data (ICLR 2024 · similarity 0.84)
• Envisioning Outlier Exposure by Large Language Models for Out-of-Distribution Detection (ICML 2024 · similarity 0.84)
• Generalize or Detect? Towards Robust Semantic Segmentation Under Multiple Distribution Shifts (NeurIPS 2024 · similarity 0.83)