Research map / Detection and segmentation / Open-set and few-shot detection
ODGEN: Domain-specific Object Detection Data Generation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Stable Diffusion on domain detection data (crops plus whole images) and controls a ControlNet with per-object text and image lists to synthesize labeled images for training YOLO detectors.
Problem
Diffusion-based synthetic data for object detection struggles with domain-specific datasets, multi-class scenes, dense and occluded objects, and objects being omitted or merged in generated images.
Approach
Fine-tune Stable Diffusion on both foreground object crops and entire images, then condition ControlNet on object-wise inputs: per-object CLIP text embeddings encoded separately and a channel-stacked image list of pasted synthesized object patches, plus a foreground/background classifier that filters pseudo labels whose objects were not generated.
Key results
- On RF7, adding 5000 ODGEN synthetic images to 200 real images raises YOLOv5s/YOLOv7 mAP@.50:.95 up to 25.3% over the real-only baseline, outperforming ReCo, GLIGEN, ControlNet and GeoDiffusion.
- On COCO, ODGEN achieves FID 16.16 versus 18.36 for the best baseline (ReCo) using 41k synthetic images.
- Adding 20k ODGEN synthetic images to 80k COCO images improves mAP@.50:.95 by 0.5% (33.1/45.9 vs 32.6/45.4 baseline) for YOLOv5s/YOLOv7.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add ODGEN-generated synthetic images to detector training | YOLOv5s and YOLOv7 · RF7 (Apex Game, Robomaster, MRI Image, Cotton, Road Traffic, Aquarium, Underwater) | mAP@.50:.95: 39.9 / 52.6 vs 38.3 / 47.2; 39.6 / 34.7 vs 27.2 / 26.5; 46.1 / 41.5 vs 37.6 / 27.4; 42.0 / 43.2 vs 16.7 / 20.5; 39.2 / 4 | Helped |
| Add 20k ODGEN synthetic images to the 80k real training images | YOLOv5s and YOLOv7 · COCO-2014 | mAP@.50:.95: 32.6 / 45.4 -> 33.1 / 45.9 | Helped |
| Use object-wise image and text lists instead of neither list (ablation) | ODGEN · Road Traffic | mAP@.50:.95 (YOLOv5s / YOLOv7): 25.5 / 35.4 -> 39.2 / 43.8 | Helped |
| Add foreground region enhancement with gamma 25 rather than gamma 0 (ablation) | ODGEN · Aquarium | mAP@.50:.95 (YOLOv5s / YOLOv7): 30.8 / 34.5 -> 32.2 / 38.5 | Helped |
| Set foreground re-weighting gamma to 100 instead of 25 (ablation) | ODGEN · Road Traffic | mAP@.50:.95 (YOLOv5s / YOLOv7): 39.2 / 43.8 -> 32.9 / 37.1 | Hurt |
| Add corrupted-label filtering (ablation) | ODGEN · Cotton | mAP@.50:.95 (YOLOv5s / YOLOv7): 40.5 / 42.1 -> 42.0 / 43.2 | Helped |
| Add foreground region enhancement (ablation) | ODGEN · COCO (10k synthetic train, real validation) | mAP@.50:.95 (YOLOv5s / YOLOv7): 8.72 / 13.50 -> 9.70 / 14.20 | Helped |
| Add cropped foreground objects to fine-tuning alongside entire images (ablation) | ODGEN diffusion model · Robomaster foreground objects | FID: 384.94 -> 124.45; 532.56 -> 136.27; 364.46 -> 135.66; 325.79 -> 146.50; 340.11 -> 128.95 | Helped |
| Train detectors using synthetic images only rather than 200 real images only (ablation) | YOLOv5s and YOLOv7 · MRI Image | mAP@.50:.95: 37.6 / 27.4 -> 41.3 / 35.3 | Helped |
| Train detectors using synthetic images only rather than 200 real images only (ablation) | YOLOv5s and YOLOv7 · Robomaster | mAP@.50:.95: 27.2 / 26.5 -> 18.7 / 21.0 | Hurt |
| Increase synthetic training set from 0 to 5000 images (ablation) | YOLOv5s and YOLOv7 · Robomaster, Cotton, Aquarium | mAP@.50: 50.3 / 48.8 -> 67.4 / 65.4; 29.4 / 32.0 -> 63.9 / 55.0; 53.8 / 53.0 -> 62.6 / 66.5 | Helped |
| Split synthesis into single-category images instead of generating mixed-category images (ablation) | ODGEN dataset synthesis · Road Traffic, Aquarium, Underwater Object | mAP@.50 (YOLOv5s / YOLOv7): 66.8 / 70.4 -> 63.4 / 70.3; 62.6 / 66.5 -> 57.2 / 63.3; 40.0 / 44.8 -> 41.1 / 42.0 | Hurt |
Surprising findings
- Detectors trained only on synthetic data outperform real-only 200-image training on easy datasets (MRI, Cotton) but not on complex ones (Robomaster, Aquarium).
- Generating each category in isolated images hurts performance in most cases, contrary to the expectation that avoiding concept bleeding would help.
Limitations
- Synthetic-only training still falls behind real-data training; VAE limits text rendering in images (e.g., Robomaster, video game text).
- Dataset synthesis pipeline (box distribution sampling) is not fully optimized; the corrupted label filter contributes only a small share of gains.
Experimental setup
Datasets: Roboflow-100 RF7 subsets (Apex Game, Robomaster, MRI Image, Cotton, Road Traffic, Aquarium, Underwater) COCO-2014
Benchmarks: Roboflow-100 RF7 COCO-2014
Metrics: FID mAP@.50 mAP@.50:.95
Baselines: ReCo GLIGEN ControlNet GeoDiffusion MIGC InstanceDiffusion
Builds on: ControlNet Stable Diffusion GLIGEN ReCo GeoDiffusion DreamBooth
Closest papers by meaning
- InstaGen: Enhancing Object Detection by Training on Synthetic Dataset (CVPR 2024 · similarity 0.87)
- Generative Region-Language Pretraining for Open-Ended Object Detection (CVPR 2024 · similarity 0.87)
- GOOD: Training-Free Guided Diffusion Sampling for Out-of-Distribution Detection (NeurIPS 2025 · similarity 0.87)
- Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detection (CVPR 2025 · similarity 0.87)
- Object-Aware Domain Generalization for Object Detection (AAAI 2024 · similarity 0.86)
- DiffuBox: Refining 3D Object Detection with Point Diffusion (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.