Research map / Detection and segmentation / Open-set and few-shot detection
SNIDA: Unlocking Few-Shot Object Detection with Non-linear Semantic Decoupling Augmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SNIDA, an instance-level data augmentation for few-shot object detection that decouples foreground and background via saliency masks and uses a semantic-guided masked autoencoder for non-linear reconstruction, plugged into DeFRCN and MFD.
Problem
Few-shot object detection overfits when only a few annotated instances per novel class exist, and conventional linear image-level augmentations (flip, rotate, scale, cutout) lack semantic awareness and diversity.
Approach
Decouple foreground and background of few-shot instances using unsupervised saliency masks to paste instances onto varied backgrounds, and generate non-linear instance-level variations by masking foreground patches and reconstructing them with an MAE whose encoder is constrained by CLIP text embeddings of novel class labels.
Key results
- On PASCAL VOC, SNIDA-MFD reaches 60.7 mean nAP50 across 3 splits and 1-10 shots, improving MFD by 1.7 on average.
- On MS-COCO, SNIDA-MFD improves MFD AP by about 11% at 1 shot, 11% at 2 shot, and 5% at 30 shot (relative gains as stated by the authors).
- SNIDA-DeFRCN raises DeFRCN's mean nAP50 on VOC from 51.9 to 55.1; ablation shows CutMix+SAES+OREM reaches 55.1 vs 52.8 for CutMix alone on split-averaged means.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SNIDA augmentation | DeFRCN · PASCAL VOC, 3-split mean | nAP50: 51.9 -> 55.1 | Helped |
| Apply SNIDA augmentation | MFD · PASCAL VOC, 3-split mean | nAP50: 59.0 -> 60.7 | Helped |
| Apply SNIDA augmentation | MFD · MS-COCO, 1, 2, 3, 5, 10, and 30 shots | AP: 10.8 -> 12.0; 13.9 -> 15.4; 15.0 -> 16.4; 16.4 -> 17.8; 19.4 -> 20.7; 22.7 -> 23.8 | Helped |
| Add CutMix (ablation) | DeFRCN · PASCAL VOC, 3-split mean | nAP50: 51.6 -> 52.8 | Helped |
| Add SAES to DeFRCN with CutMix (ablation) | DeFRCN · PASCAL VOC, 3-split mean | nAP50: 52.8 -> 53.9 | Helped |
| Add OREM to DeFRCN with CutMix (ablation) | DeFRCN · PASCAL VOC, 3-split mean | nAP50: 52.8 -> 54.2 | Helped |
| Add both SAES and OREM to DeFRCN with CutMix (ablation) | DeFRCN · PASCAL VOC, 3-split mean | nAP50: 52.8 -> 55.1 | Helped |
| Replace correct semantic guidance with random labels (ablation) | OREM · PASCAL VOC, Novel Split 1 | nAP50 mean: 63.1 -> 60.2 | Hurt |
| Replace correct semantic guidance with incorrect labels (ablation) | OREM · PASCAL VOC, Novel Split 1 | nAP50 mean: 63.1 -> 60.8 | Hurt |
| Use correct semantic guidance (ablation) | OREM · PASCAL VOC, Novel Split 1 | nAP50 mean: 61.7 -> 63.1 | Helped |
Surprising findings
- Finetuning with the proposed augmentation is about 31% slower per iteration than the baseline (0.63 vs 0.48 s/iter), a stated drawback.
Limitations
- Finetuning is roughly 31% slower than the baseline.
- Relies on unsupervised saliency detection and CLIP label embeddings for semantic guidance, which may be noisy for few-shot objects.
- Gains over strong baselines are modest on some splits and shots (e.g., VOC 10-shot for DeFRCN is slightly lower than baseline on some splits).
Experimental setup
Datasets: PASCAL VOC MS-COCO ImageNet-1k
Benchmarks: PASCAL VOC MS-COCO
Metrics: nAP50 AP
Baselines: DeFRCN MFD TFA FSCE Norm-VAE LVC Cutout GridMask CutMix
Builds on: DeFRCN MFD CutMix MAE TFA
Closest papers by meaning
- Decoupling Classifier for Boosting Few-shot Object Detection and Instance Segmentation (NeurIPS 2022 · similarity 0.88)
- Few-Shot Object Detection via Classification Refinement and Distractor Retreatment (CVPR 2021 · similarity 0.88)
- Multi-Perspective Data Augmentation for Few-shot Object Detection (ICLR 2025 · similarity 0.87)
- Weakly Supervised Few-Shot Object Detection with DETR (AAAI 2024 · similarity 0.85)
- Fine-Grained Prototypes Distillation for Few-Shot Object Detection (AAAI 2024 · similarity 0.85)
- Revisiting Few-Shot Object Detection with Vision-Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.