Research map / Detection and segmentation / Open-set and few-shot detection
Test-Time Adaptive Object Detection with Foundation Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a multi-modal prompt-based mean-teacher framework on Grounding DINO for source-free, open-vocabulary test-time adaptive object detection, with a test-time warm-start for visual prompts and an instance dynamic memory (memory enhancement and hallucination).
Problem
Test-time adaptive object detection currently needs source-domain statistics and assumes an identical closed label space, and adapting vision-language detectors at test time risks overfitting and unreliable pseudo-labels.
Approach
Adapt Grounding DINO at test time by tuning only text and visual prompts in a mean-teacher setup, initialize visual prompts by average-pooling first-sample image tokens, and keep a per-class memory of high-quality DINOv2-encoded instance crops to refine predictions and hallucinate positives in images without pseudo-labels.
Key results
- On Pascal-C, the method reaches 56.2% average AP50 vs 51.5% for Mean-Teacher on Grounding DINO and 45.2% for prior SOTA STFAR.
- On COCO-C it reaches 26.0% average mAP, the highest among compared methods, without source data.
- On ODinW-13 it reaches 54.2% average mAP vs 52.8% for direct test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply the full multimodal prompt and memory method instead of Grounding DINO Mean-Teacher | Grounding DINO · Pascal-C | Avg AP50: 51.5% vs. 56.2% | Helped |
| apply the full multimodal prompt and memory method instead of Grounding DINO Mean-Teacher | Grounding DINO · COCO-C | Avg mAP: 24.1 vs 26.0 | Helped |
| apply the full method instead of Direct Test | Grounding DINO · ODinW-13 | Avg mAP: 52.8% to 54.2%; 1.4% | Helped |
| use text prompt tuning only (ablation) | Grounding DINO · Pascal-C | Avg AP50: 44.8 -> 45.4; 0.6% | Helped |
| add visual prompt tuning without warm-start (ablation) | Grounding DINO · Pascal-C | Avg AP50: 44.8 -> 41.4; 3.4% | Hurt |
| add Test-time Warm-start to visual prompt tuning (ablation) | Grounding DINO · Pascal-C | Avg AP50: 41.4 -> 53.4 | Helped |
| add text prompt tuning to visual prompt tuning with Test-time Warm-start (ablation) | Grounding DINO · Pascal-C | Avg AP50: 53.4 -> 53.9 | Helped |
| add Memory Hallucination to MPMT (ablation) | Grounding DINO MPMT · Pascal-C | Avg AP50: 53.9 -> 54.9; 1.0% | Helped |
| add Memory Enhancement to MPMT with Memory Hallucination (ablation) | Grounding DINO · Pascal-C | Avg AP50: 54.9 -> 56.2 | Helped |
| increase visual prompt count from 2 to 10 (ablation) | Grounding DINO visual prompt tuning · Pascal-C Gaussian noise | AP50: 42.6 -> 45.4 | Helped |
| increase visual prompt count from 10 to 50 (ablation) | Grounding DINO visual prompt tuning · Pascal-C Gaussian noise | AP50: 45.4 -> 44.2 | Hurt |
Surprising findings
- Visual prompt tuning alone degrades performance (41.4% vs 44.8% direct test on Pascal-C) and can cause catastrophic failure on some corruptions, which the warm-start fixes.
- Text prompt tuning alone yields only a small gain (45.4% vs 44.8%).
Limitations
- Adding Memory Enhancement and Hallucination increases inference latency compared with direct testing.
- Gains are small or negative on datasets with very few test samples (Mu, Pa in ODinW-13).
- No explicit limitations section in the paper.
Experimental setup
Datasets: Pascal-C COCO-C ODinW-13 Objects365 GoldG Cap4M
Benchmarks: Pascal-C COCO-C ODinW-13
Metrics: AP50 mAP latency memory
Baselines: Direct Test BN TENT T3A SHOT Mean-Teacher STFAR CTTAOD CTTAOD-Skip
Builds on: Grounding DINO Mean-Teacher TDA VPT STFAR
Closest papers by meaning
- What How and When Should Object Detectors Update in Continually Changing Test Domains? (CVPR 2024 · similarity 0.89)
- Efficient Test-Time Adaptation of Vision-Language Models (CVPR 2024 · similarity 0.89)
- BoostAdapter: Improving Vision-Language Test-Time Adaptation via Regional Bootstrapping (NeurIPS 2024 · similarity 0.89)
- DOTA: Distributional Test-time Adaptation of Vision-Language Models (NeurIPS 2025 · similarity 0.88)
- Noisy Test-Time Adaptation in Vision-Language Models (ICLR 2025 · similarity 0.88)
- Bayesian Test-Time Adaptation for Vision-Language Models (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.