Research map / Detection and segmentation / Open-set and few-shot detection
Generalized Diffusion Detector: Mining Robust Features from Diffusion Models for Domain-Generalized Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Uses multi-timestep intermediate features from a frozen Stable Diffusion model as a domain-robust detector backbone, then distills its feature and object-level knowledge into a conventional ResNet Faster R-CNN to improve domain-generalized object detection without extra inference cost.
Problem
Domain generalization for object detection: detectors trained on a source domain degrade substantially on unseen target domains (weather, camera, synthetic-to-real, artistic styles, corruptions) without access to target data.
Approach
Extract and fuse multi-scale features from several noising/denoising timesteps of a frozen diffusion model to build a diffusion-based detector, then transfer its knowledge to a standard detector via correlation-based feature alignment (PKD) and shared-RoI object-level distillation (CrossKD-style), keeping inference cost unchanged.
Key results
- Diff. Guided (SD-1.5) reaches 46.3 mAP on BDD100K cross-camera (+20.9 over baseline), surpassing DA method HT (40.2) without target data.
- On the Diverse Weather benchmark, Diff. Guided (SD-1.5) achieves 38.1 average (+13.9 over baseline) versus 35.5 for DivAlign.
- Diff. Detector (SD-1.5) reaches 58.3 on Clipart and 62.8 mAP on Cityscapes-to-Car synthetic-to-real settings, but gains are limited on the Real-to-Artistic benchmark.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use diffusion features with SD-1.5 instead of the strongest listed DG baseline | Domain-generalized detector · BDD100K | mAP: 28.0 -> 46.6 | Helped |
| Use diffusion features with SD-1.5 instead of the strongest listed DG baseline | Domain-generalized detector · FoggyCityscapes | mAP: 38.6 -> 50.1 | Helped |
| Use diffusion features with SD-1.5 instead of the strongest listed DG baseline | Domain-generalized detector · RainyCityscapes | mAP: 42.8 -> 58.2 | Helped |
| Use diffusion guidance instead of Faster R-CNN baseline | Faster R-CNN · Cityscapes (car), synthetic to real | AP: +22.3 | Helped |
| Use diffusion features with SD-1.5 instead of the strongest listed DG baseline | Domain-generalized detector · Diverse Weather average | AP: 35.5 -> 40.2 | Helped |
| Use diffusion guidance instead of Faster R-CNN baseline | Faster R-CNN · Clipart | AP: 49.3 -> 40.8 | Hurt |
| Use diffusion guidance instead of Faster R-CNN baseline | Faster R-CNN · Comic | AP: 40.5 -> 29.7 | Hurt |
| Use diffusion guidance instead of Faster R-CNN baseline | Faster R-CNN · Watercolor | AP: 59.9 -> 54.2 | Hurt |
| Use diffusion features instead of the strongest listed baseline | Domain-generalized detector · Cityscapes corruption | mPC: 21.8 -> 24.1 | Helped |
| Add feature-level alignment (ablation) | Diff. Guided detector · BDD100K | mAP: 29.8+4.4 -> 38.1+12.7 | Helped |
| Add object-level alignment (ablation) | Diff. Guided detector · BDD100K | mAP: 29.8+4.4 -> 34.2+9.5 | Helped |
| Increase diffusion steps from T=1 to T=10 (ablation) | Diff. Detector · BDD100K | mAP: 28.6 -> 47.1 | Helped |
Surprising findings
- Without noise diffusion (max timestep 0), the diffusion detector still transfers strongly, similar to GLIP, suggesting gains partly stem from large-scale pre-training.
- Domain augmentation gives only 2.7% gain for the diffusion detector but 11.0% for the ResNet baseline.
Limitations
- Diffusion detector has large parameter count and multi-step denoising cost, limiting scalability.
- Gains are limited under extreme domain shifts such as Real-to-Artistic.
- Feature/object alignment trades off source fit against target generalization.
Experimental setup
Datasets: Cityscapes BDD100K FoggyCityscapes RainyCityscapes Sim10K VOC Clipart Comic Watercolor Daytime-Sunny Night-Sunny Night-Rainy
Benchmarks: BDD100K FoggyCityscapes RainyCityscapes Cityscapes Clipart Comic Watercolor Diverse Weather benchmark Cityscapes Corruption benchmark
Metrics: mAP AP50 mPC rPC D-ECE
Baselines: CDSD SHADE SRCD MAD FACT FSDR OA-DG DivAlign UFR CLIPGap HT CMT
Builds on: Stable Diffusion Faster R-CNN PKD CrossKD DDIM
Closest papers by meaning
- Object-Aware Domain Generalization for Object Detection (AAAI 2024 · similarity 0.89)
- DiffuBox: Refining 3D Object Detection with Point Diffusion (NeurIPS 2024 · similarity 0.89)
- Towards Single-Source Domain Generalized Object Detection via Causal Visual Prompts (NeurIPS 2025 · similarity 0.89)
- Improving Single Domain-Generalized Object Detection: A Focus on Diversification and Alignment (CVPR 2024 · similarity 0.89)
- Unbiased Faster R-CNN for Single-source Domain Generalized Object Detection (CVPR 2024 · similarity 0.88)
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.