Research map / Video / Video segmentation and tracking
QDETRv: Query-Guided DETR for One-Shot Object Localization in Videos
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends DETR with query-image cross-attention over temporal context frames and a video-specific unsupervised pretraining on synthetic trajectories for one-shot object localization in videos.
Problem
One-shot, class-agnostic localization of unseen objects in videos given a single query image, where the model must exploit spatio-temporal context.
Approach
Adds a cross-attention module that uses query-image features against neighboring-frame context to produce context-aware features fed into DETR's encoder, plus a video pretraining on synthetic trajectories with recurrent object queries and patch feature reconstruction loss.
Key results
- With video pretraining, QDETRv reaches 43.1 mAP (seen) and 41.6 mAP (unseen) on VidOR, versus UP-DETR with pretraining at 41.7 and 38.1.
- On VidVRD, QDETRv with pretraining reaches 42.6 (seen) and 38.5 (unseen) mAP, above UP-DETR with pretraining at 39.4 and 37.1.
- Increasing the temporal context window from 3 to 9 frames monotonically improves mAP on VidOR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add QDETRv's query-guided temporal context module (ablation) | UP-DETR · VidOR Seen | mAP: 37.4 -> 40.2 | Helped |
| add QDETRv's query-guided temporal context module (ablation) | UP-DETR · VidOR Unseen | mAP: 35.4 -> 38.6 | Helped |
| add QDETRv's query-guided temporal context module (ablation) | UP-DETR · VidVRD Seen | mAP: 33.7 -> 38.9 | Helped |
| add QDETRv's query-guided temporal context module (ablation) | UP-DETR · VidVRD Unseen | mAP: 28.5 -> 35.7 | Helped |
| add video-specific unsupervised pretraining (ablation) | QDETRv · VidOR Seen | mAP: 40.2 -> 43.1 | Helped |
| add video-specific unsupervised pretraining (ablation) | QDETRv · VidOR Unseen | mAP: 38.6 -> 41.6 | Helped |
| add video-specific unsupervised pretraining (ablation) | QDETRv · VidVRD Seen | mAP: 38.9 -> 42.6 | Helped |
| add video-specific unsupervised pretraining (ablation) | QDETRv · VidVRD Unseen | mAP: 35.7 -> 38.5 | Helped |
| add recurrent object queries during pretraining (ablation) | QDETRv · VidOR | mAP: 40.3 -> 41.6 | Helped |
| add recurrent object queries during pretraining (ablation) | QDETRv · VidVRD | mAP: 37.9 -> 38.5 | Helped |
| increase the temporal context window size (ablation) | QDETRv · VidOR | mAP | Helped |
Surprising findings
- Recurrent queries during pretraining add only modest gains (0.6 to 1.3 mAP), and the paper's reported ablation numbers are inconsistent between text and tables.
Limitations
- Relies heavily on query image quality; occluded or partial query views lead to weak localization.
- Fails on occluded, small, overlapping, and visually similar objects.
Experimental setup
Datasets: VidOR ImageNet-VidVRD Open Images UCF101 ImageNet
Benchmarks: VidOR ImageNet-VidVRD
Metrics: mAP@0.5
Baselines: CoAE Retrieval-Based OWL-ViT UP-DETR
Builds on: DETR UP-DETR CoAE
Closest papers by meaning
- RELOCATE: A Simple Training-Free Baseline for Visual Query Localization Using Region-Based Representations (CVPR 2025 · similarity 0.87)
- Context Enhanced Transformer for Single Image Object Detection in Video Data (AAAI 2024 · similarity 0.86)
- Dual DETRs for Multi-Label Temporal Action Detection (CVPR 2024 · similarity 0.85)
- Transferable Video Moment Localization by Moment-Guided Query Prompting (AAAI 2024 · similarity 0.85)
- Dense Video Object Captioning from Disjoint Supervision (ICLR 2025 · similarity 0.85)
- Knowing Your Target: Target-Aware Transformer Makes Better Spatio-Temporal Video Grounding (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.