Research map / Detection and segmentation / Open-set and few-shot detection
Multi-Perspective Data Augmentation for Few-shot Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MPAD, a multi-perspective diffusion-based data augmentation framework (ICOS prompts from ChatGPT, HPAS prompt-embedding mixing, BAP background selection) for few-shot object detection on PASCAL VOC and MS COCO.
Problem
Few-shot object detection suffers from overfitting and limited diversity of augmented novel-class samples; existing diffusion-based augmentation uses simple prompts and ignores foreground-background relations.
Approach
Combines LLM in-context prompt diversification (attributes and fine-grained categories), time-step prompt embedding mixing between novel and base classes to create hard samples, and typical/hard background selection (entropy clutter and visual similarity) to synthesize diverse training images for a DeFRCN detector.
Key results
- On PASCAL VOC, MPAD reaches 66.9 mean nAP50 across three novel sets, averaging a 17.5-point gain over the DeFRCN baseline.
- On MS COCO 1-shot, MPAD reports nAP 18.3 and nAP50 31.2, outperforming Norm-VAE (9.5 nAP) and SFOT.
- Ablation on Novel Set 1 shows ICOS raises nAP50 from 62.1 to 68.5 and HPAS adds to 69.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply MPAD data augmentation instead of the DeFRCN baseline | DeFRCN · PASCAL VOC, mean across Novel Sets 1–3 | nAP50: 61.6 -> 66.9 | Helped |
| Apply MPAD data augmentation instead of DeFRCN | DeFRCN · MS COCO, 1-shot | nAP: 4.8 -> 18.3 | Helped |
| Apply MPAD data augmentation instead of DeFRCN | DeFRCN · MS COCO, 1-shot | nAP50: 9.5 -> 31.2 | Helped |
| Apply MPAD data augmentation instead of DeFRCN | DeFRCN · MS COCO, 1-shot | nAP75: 4.4 -> 18.8 | Helped |
| Replace CutMix augmentation with MPAD | PASCAL VOC Novel Set 1, 1-shot · PASCAL VOC Novel Set 1, 1-shot | nAP50: 55.5 -> 69.1 | Helped |
| Add ICOS attribute prompts (ablation) | MPAD foreground synthesis · PASCAL VOC Novel Set 1 | nAP50: 62.1 -> 65.8 | Helped |
| Add ICOS fine-grained categories (ablation) | MPAD foreground synthesis with attributes · PASCAL VOC Novel Set 1 | nAP50: 65.8 -> 68.5 | Helped |
| Add HPAS prompt mixing (ablation) | MPAD foreground synthesis · PASCAL VOC Novel Set 1 | nAP50: 68.5 -> 69.1 | Helped |
| Select typical clutter and hard similar backgrounds instead of random backgrounds (ablation) | MPAD background selection · PASCAL VOC Novel Set 1 | nAP50: 62.1 -> 64.1 | Helped |
| Increase the number of fine-grained categories from 1 to 4 (ablation) | MPAD · PASCAL VOC Novel Set 1 | nAP50: 66.8 -> 68.5 | Helped |
| Increase fine-grained categories from 4 to 6 (ablation) | MPAD · PASCAL VOC Novel Set 1 | nAP50: 68.5 -> 64.8 | Hurt |
| Use momentum in HPAS (ablation) | MPAD · PASCAL VOC Novel Set 1 | Average of nAP, nAP50, and nAP75: about 1 to 2% | Helped |
Surprising findings
- Training only on synthetic data (scheme 1) reaches up to 65.4 nAP50 in 1-shot, while pre-training on synthetic data then fine-tuning on real data (scheme 2) degrades performance relative to the single-stage approach.
Limitations
- Diffusion models still hallucinate, producing objects unrelated to the prompt or low-quality images.
- The starting weight w in HPAS is fixed and may not suit all novel classes.
- Default DeFRCN training hyperparameters may not converge on highly diverse synthetic datasets.
Experimental setup
Datasets: PASCAL VOC MS COCO ImageNet
Benchmarks: PASCAL VOC MS COCO
Metrics: nAP50 nAP nAP75
Baselines: TFA FSDetView MPSR FSCE SRR-FSD DeFRCN Meta R-CNN MFD Norm-VAE Lin et al. SNIDA SFOT
Builds on: DeFRCN PowerPaint TFA Stable Diffusion
Closest papers by meaning
- Domain-RAG: Retrieval-Guided Compositional Image Generation for Cross-Domain Few-Shot Object Detection (NeurIPS 2025 · similarity 0.87)
- Fine-Grained Prototypes Distillation for Few-Shot Object Detection (AAAI 2024 · similarity 0.87)
- SNIDA: Unlocking Few-Shot Object Detection with Non-linear Semantic Decoupling Augmentation (CVPR 2024 · similarity 0.87)
- Few-Shot Object Detection with Foundation Models (CVPR 2024 · similarity 0.86)
- Revisiting Few-Shot Object Detection with Vision-Language Models (NeurIPS 2024 · similarity 0.86)
- Pushing the Limit of Fine-Tuning for Few-Shot Learning: Where Feature Reusing Meets Cross-Scale Attention (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.