Research map / Detection and segmentation / CNN features
TGBFormer: Transformer-GraphFormer Blender Network for Video Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TGBFormer, which fuses a spatial-temporal transformer (global features) with a graph-convolutional GraphFormer (local features) via a learned global-local blender for video object detection on ImageNet VID.
Problem
Video object detection methods rely solely on CNNs (good local, weak global) or ViTs (good global, weak local), limiting detection accuracy; frame-wise detection is also slow.
Approach
Combine a spatial-temporal transformer for long-range global features and a spatial-temporal dynamic graph convolution ("GraphFormer") with graph pruning for short-range local features, adaptively blended by a learned softmax gate, and detect all frames of a clip in parallel with DETR-style queries.
Key results
- TGBFormer with ResNet-101 reaches 86.5% mAP on ImageNet VID at 24.3 ms runtime, 1.1% above CDANet (85.4%).
- With Swin-B backbone, TGBFormer reaches 90.3% mAP at 49.7 ms, versus 90.1% for TransVOD Lite.
- Ablation: adding the global-local blender improves overall mAP from 85.4% (concatenation) to 86.5%; the transformer alone raises DETR baseline from 77.6% to 83.3%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the spatial-temporal transformer module to DETR (ablation) | DETR with ResNet-101 · ImageNet VID validation (overall) | mAP (%): 77.6% to 83.3% | Helped |
| add the spatial-temporal GraphFormer without graph pruning to DETR (ablation) | DETR with ResNet-101 · ImageNet VID validation (overall) | mAP (%): 77.6% to 82.5% | Helped |
| add graph pruning to the spatial-temporal GraphFormer (ablation) | GraphFormer variant (c) · ImageNet VID validation (overall) | mAP (%): 0.6% | Helped |
| add the spatial-temporal transformer and concatenate global and local features (ablation) | GraphFormer with graph pruning (d) · ImageNet VID validation (overall) | mAP (%): 2.3% | Helped |
| add the global-local feature blender (ablation) | Transformer + GraphFormer with concatenation (e) · ImageNet VID validation (overall) | mAP (%): 1.1% | Helped |
| increase dynamic graph convolution layers from 2 to 4 (ablation) | TGBFormer STGM · ImageNet VID validation | mAP (%): from 86.5% mAP to 85.6% mAP | Hurt |
| increase inference frames from 25 to 30 (ablation) | TGBFormer · ImageNet VID validation | mAP (%): 86.5 to 86.3 | Hurt |
| increase inference frames from 25 to 30 (ablation) | TGBFormer · ImageNet VID validation | Runtime (ms): 24.3 to 21.7 | Helped |
Surprising findings
- Using 4 dynamic graph convolution layers drops mAP to 85.6% from 86.5% at 2 layers, attributed to ambiguous connection edges.
Limitations
- Performance drops when the number of dynamic graph layers grows beyond 2.
- Accuracy declines slightly when inference clip length increases from 25 to 30 frames.
Experimental setup
Datasets: ImageNet VID ImageNet DET
Benchmarks: ImageNet VID
Metrics: mAP@0.5 runtime (ms) FPS
Baselines: DETR CDANet ClipVID TransVOD Lite QueryProp GMLCN MEGA CETR
Builds on: DETR Deformable DETR graph convolutional network ResNet
Closest papers by meaning
- MSTDiff: Multiscale-Aware Transformer Diffusion Network for Video Object Detection (AAAI 2026 · similarity 0.89)
- When Transformers Meet Mamba: A Hybrid Transformer-Mamba Network for Video Object Detection (CVPR 2026 · similarity 0.89)
- RelationNet++: Bridging Visual Representations for Object Detection via Transformer Decoder (NeurIPS 2020 · similarity 0.89)
- Object-Region Video Transformers (CVPR 2022 · similarity 0.88)
- Context Enhanced Transformer for Single Image Object Detection in Video Data (AAAI 2024 · similarity 0.87)
- SparseFormer: Sparse Visual Recognition via Limited Latent Tokens (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.