Research map / Detection and segmentation / Tracking
TGFormer: Transformer with Track Query Group for Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TGFormer, a transformer multi-object tracker that assigns each target a group of track queries (one per occlusion level), updated by a memory-based updater and a LSTM position predictor.
Problem
Single track query per target in transformer-based MOT is brittle under varying occlusion levels and appearance changes in crowded scenes, causing ID switches.
Approach
Replace the single track query with a Track Query Group per target, with queries specialized to fully exposed, slightly occluded, and heavily occluded states; update them with a short-term memory aggregator, occlusion-aware memory attention, and long-term memory residuals; and predict target position with an LSTM.
Key results
- On MOT17 test, TGFormer reaches 74.9 MOTA and 60.3 HOTA among transformer-based trackers, versus 72.8 MOTA and 58.8 HOTA for MeMOTR.
- On DanceTrack test, TGFormer reaches 91.3 MOTA, 71.9 IDF1 and 69.1 HOTA, above MeMOTR's 89.9 / 71.2 / 68.5.
- On MOT20 test, TGFormer reaches 70.3 MOTA, 67.1 IDF1 and 54.2 HOTA, exceeding MeMOT's 63.7 / 66.1 / 54.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Track Query Group and Short-Term Memory Aggregator (ablation) | MeMOTR · MOT17 validation | MOTA: 67.4 -> 72.1 | Helped |
| add Track Query Group and Short-Term Memory Aggregator (ablation) | MeMOTR · MOT17 validation | IDF1: 66.4 -> 66.5 | No clear effect |
| add Track Query Group and Short-Term Memory Aggregator (ablation) | MeMOTR · MOT17 validation | HOTA: 54.6 -> 57.9 | Helped |
| add Occlusion-Aware Memory-Attention and long-term memory to the aggregator (ablation) | TGFormer · MOT17 validation | MOTA: 72.1 -> 74.3 | Helped |
| add Occlusion-Aware Memory-Attention and long-term memory to the aggregator (ablation) | TGFormer · MOT17 validation | IDF1: 66.5 -> 67.9 | Helped |
| add Occlusion-Aware Memory-Attention and long-term memory to the aggregator (ablation) | TGFormer · MOT17 validation | HOTA: 57.9 -> 58.3 | Helped |
| use 3 input frames and 1 prediction frame for Position Predictor (ablation) | TGFormer · MOT17 validation | MOTA: 74.1 | Helped |
| use 5 input frames and 2 prediction frames for Position Predictor (ablation) | TGFormer · MOT17 validation | MOTA: 74.3 | Helped |
| use 10 input frames and 3 prediction frames for Position Predictor (ablation) | TGFormer · MOT17 validation | MOTA: 74.9 | Helped |
| use 10 input frames and 3 prediction frames rather than 5 input frames and 2 prediction frames (ablation) | TGFormer · MOT17 validation | IDF1: 67.8 -> 68.3 | Helped |
| use Track Query Group and its updater | MeMOTR · MOT17 test | HOTA: 58.8 -> 60.3 | Helped |
| use TGFormer rather than MeMOTR | transformer-based tracking on MOT20 · MOT20 test | MOTA | Helped |
Surprising findings
- Ablation shows the Track Query Group mainly boosts MOTA (+4.7) with almost no IDF1 gain (+0.1) when only the short-term aggregator is applied.
Limitations
- Ablations use a split of MOT17 training sequences rather than the official test set.
- Relies on confidence thresholds (0.85, 0.7, 0.5) hand-set for occlusion-level categories.
- Position predictor needs a two-stage training process and the history length limits its input.
Experimental setup
Datasets: MOT17 MOT20 DanceTrack CrowdHuman
Benchmarks: MOT17 MOT20 DanceTrack
Metrics: MOTA IDF1 HOTA AssA DetA MT ML IDS
Baselines: TrackFormer MOTR TransTrack TransCenter MeMOT MeMOTR MOTRv2 FairMOT ByteTrack GeneralTrack CenterTrack
Builds on: MeMOTR DAB-DETR MOTR MeMOT
Closest papers by meaning
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.89)
- Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking (ICLR 2025 · similarity 0.88)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.87)
- S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking (ICML 2025 · similarity 0.87)
- More Than Meets the Eye: Enhancing Multi-Object Tracking Even with Prolonged Occlusions (ICML 2025 · similarity 0.86)
- Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.