Research map / Detection and segmentation / Open-set and few-shot detection
SAUI: Scale-Aware Unseen Imagineer for Zero-Shot Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAUI, a generation-based zero-shot object detector that extracts multi-scale (coarse-to-fine) RoI features and synthesizes unseen features scale-by-scale with a Series-GAN plus three multi-scale contrastive losses.
Problem
Zero-shot object detection generators synthesize unseen features at a single scale, ignoring the scale-varying nature of objects, which limits scale diversity and hurts detection of scale-varying and small unseen objects.
Approach
Adds scale-view RoI pooling channels across FPN levels to get multi-scale features, and a Series-GAN synthesizer that generates features coarse-to-fine as a sequence, trained with intra-scale outline/detail and inter-scale contrastive losses.
Key results
- On COCO 65/15 ZSD, SAUI reaches 22.1% mAP and 77.8% Recall@100 (IoU=0.5), above RSC's 20.1% mAP.
- On DIOR 16/4, SAUI reaches 16.1% ZSD mAP and 17.1% GZSD mAP-HM, versus 11.3%-ish mAP for the next best generation method (RRFS, 6.1 HM).
- On COCO 65/15, SAUI obtains 9.9% APs for small objects, roughly 3 times the second-best score.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scale-aware SAUI | Faster R-CNN ResNet-101 · COCO 65/15 ZSD | mAP: 20.1 -> 22.1 | Helped |
| add scale-aware SAUI | Faster R-CNN ResNet-101 · DIOR 16/4 ZSD | mAP: 11.3 -> 16.6 | Helped |
| add Series-GAN instead of vanilla GAN (ablation) | SAUI on DIOR · DIOR 16/4 ZSD | mAP: 12.2 -> 16.6 | Helped |
| use CLIP word embeddings (ablation) | SAUI on DIOR · DIOR 16/4 ZSD | mAP: 15.5 -> 16.6 | Helped |
| add Intra-scale Outline Maintaining loss (ablation) | SAUI on DIOR · DIOR 16/4 ZSD | mAP: 15.9 -> 16.6 | Helped |
| add Intra-scale Detail Diverging loss (ablation) | SAUI on DIOR · DIOR 16/4 ZSD | mAP: 15.7 -> 16.6 | Helped |
| add Inter-scale Mutual Constructing loss (ablation) | SAUI on DIOR · DIOR 16/4 ZSD | mAP: 15.5 -> 16.6 | Helped |
Surprising findings
- On PASCAL VOC, SAUI performs comparably rather than best, which the authors attribute to large, prominent objects where the scale-aware design degrades to vanilla methods.
Limitations
- Limited gains on PASCAL VOC where objects are large and prominent.
- Requires training four scale-view channels and per-scale generator/discriminator pairs, adding complexity.
- Results on GZSD mAP for COCO 65/15 are not uniformly best (SU has higher seen mAP in some cases).
Experimental setup
Datasets: PASCAL VOC 2007+2012 MS COCO 2014 DIOR
Benchmarks: PASCAL VOC MS COCO DIOR
Metrics: mAP Recall@100 harmonic mean (HM) APs/APm/APl
Baselines: PL BLC ConZSD RRFS TCB SU RSC TZSDC SAN HRE
Builds on: Faster R-CNN Feature Pyramid Network Wasserstein GAN SU (synthesizing unseen features) RRFS
Closest papers by meaning
- Don't Even Look Once: Synthesizing Features for Zero-Shot Detection (CVPR 2020 · similarity 0.90)
- Context-Aware Zero-Shot Learning for Object Recognition (ICML 2019 · similarity 0.87)
- Revisiting Few-Shot Object Detection with Vision-Language Models (NeurIPS 2024 · similarity 0.85)
- Percept, Memory, and Imagine: World Feature Simulating for Open-Domain Unknown Object Detection (CVPR 2025 · similarity 0.84)
- Roboflow100-VL: A Multi-Domain Object Detection Benchmark for Vision-Language Models (NeurIPS 2025 · similarity 0.84)
- PerceptionCLIP: Visual Classification by Inferring and Conditioning on Contexts (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.