Research map / Detection and segmentation / CNN features
Context Enhanced Transformer for Single Image Object Detection in Video Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CETR, a single-image DETR-based detector that injects dataset-level temporal context via a class-wise prototype memory module, score-based memory sampling, and test-time memory adaptation for video object detection.
Problem
Video object detection usually relies on multiple adjacent or sampled frames, adding computational cost and limiting real-time practicality; the paper asks how to exploit temporal context while keeping single-image inference.
Approach
A non-parametric, fixed-size class-wise multi-prototype memory updated by momentum over the whole dataset is read by the Transformer encoder, sampled per image by a classification-score-based multi-threshold module, and queried by a memory-guided decoder; memory is further adapted at test time with per-memory update rates.
Key results
- On CityCam, Deformable DETR + CETR reaches 25.7 AP and 43.0 AP50 versus 24.2 AP and 41.0 AP50 for the Deformable DETR baseline, at 30.6 FPS versus 37.9 FPS.
- On ImageNet VID, CETR achieves 79.6 AP50 among online methods at 23.3 FPS with 0.55 GB memory, the highest online AP50 in the comparison table.
- Ablation: score-based sampling plus multi-threshold sampling raises AP50 from 40.2 (baseline) to 42.5 on CityCam; ground-truth sampling oracle reaches 57.6 AP50.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CETR to Deformable DETR | Deformable DETR · CityCam test | AP: 24.2 -> 25.7 | Helped |
| Add CETR to Deformable DETR | Deformable DETR · CityCam test | FPS: 37.9 -> 30.6 | Hurt |
| Use CETR for single-frame video object detection | ResNet-101 video object detector · ImageNet VID validation | AP50: 81.8 -> 79.6 | Hurt |
| Add CMM (ablation) | DAB-DETR · CityCam test | AP50: 40.2 -> 40.5 | Helped |
| Add MGD without score-based sampling or multi-level thresholding (ablation) | DAB-DETR with CMM · CityCam test | AP50: 40.5 -> 40.5 | No clear effect |
| Add score-based sampling and multi-level thresholding (ablation) | DAB-DETR with CMM and MGD · CityCam test | AP50: 40.5 -> 42.5 | Helped |
| Use 1 prototype per class instead of 3 (ablation) | CETR · CityCam test | AP50: 42.5 -> 41.9 | Hurt |
| Use 5 or 10 prototypes per class instead of 3 (ablation) | CETR · CityCam test | AP50: 42.5 -> 41.1; 42.5 -> 40.0 | Hurt |
| Replace score-based sampling with learnable, full, or random memory sampling (ablation) | CETR · CityCam test | AP50: 42.5 -> 40.9; 42.5 -> 40.5; 42.5 -> 41.2 | Hurt |
| Replace score-based sampling with oracle ground-truth sampling (ablation) | CETR · CityCam test | AP50: 42.5 -> 57.6 | Helped |
| Add test-time memory update (ablation) | CETR · CityCam test | AP50: 42.5 -> 42.8 | Helped |
| Add camera-specific memory adaptation (ablation) | CETR with test-time memory update · CityCam test | AP50: 42.8 -> 43.0 | Helped |
Surprising findings
- Using only a single prototype per class, or more than 3 prototypes, gives lower gains on CityCam, suggesting too many prototypes hinder class feature use.
- The memory-guided decoder alone gives no improvement; gains appear only when combined with score-based sampling.
Limitations
- CityCam experiments use only vehicle classes, so prototype benefits may not generalize to diverse categories.
- Reported gains on ImageNet VID are compared against methods with differing backbones and settings; FPS trades off against DETR variant choice.
Experimental setup
Datasets: CityCam ImageNet VID ImageNet DET COCO
Benchmarks: CityCam ImageNet VID
Metrics: AP AP50 APS APM APL FPS GPU memory (GB)
Baselines: Faster R-CNN Conditional DETR DAB-DETR Deformable DETR TransVOD SELSA LRTR RDN OGEMN LSTS
Builds on: DETR DAB-DETR Deformable DETR asymmetric loss
Closest papers by meaning
- Temporal Object-Aware Vision Transformer for Few-Shot Video Object Detection (AAAI 2026 · similarity 0.89)
- TGBFormer: Transformer-GraphFormer Blender Network for Video Object Detection (AAAI 2025 · similarity 0.87)
- QDETRv: Query-Guided DETR for One-Shot Object Localization in Videos (AAAI 2024 · similarity 0.86)
- ContextShift: A Controlled Benchmark for Context Dependence in Object Detection (NeurIPS 2026 · similarity 0.85)
- Scene Adaptive Sparse Transformer for Event-based Object Detection (CVPR 2024 · similarity 0.85)
- Test-Time Adaptive Object Detection with Foundation Model (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.