Research map / Detection and segmentation / Tracking
Samba: Synchronized Set-of-Sequences Modeling for Multiple Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Samba, a linear-time set-of-sequences model that synchronizes per-tracklet selective SSM (Mamba) states via self-attention, and plugs it into a tracking-by-propagation tracker (SambaMOTR) with MaskObs and a long-sequence training recipe to reach SOTA on DanceTrack, BFT and SportsMOT.
Problem
Multi-object tracking in complex scenes with coordinated motion, occlusions, and long-range trajectory dependencies; prior tracking-by-propagation methods only propagate across adjacent frames and model tracklets independently.
Approach
Synchronize the hidden states of multiple selective state-space models (one per tracklet) at each time step with multi-head self-attention, enabling tracklet interaction while keeping linear time in sequence length; mask unreliable observations from the SSM state update (MaskObs); train on 10-frame clips with gradients on the last 5.
Key results
- SambaMOTR reaches 67.2 HOTA and 57.5 AssA on DanceTrack test, +3.8 HOTA and +5.2 AssA over MeMOTR (63.4 / 52.3).
- On BFT, SambaMOTR achieves 69.6 HOTA, +2.8 over OC-SORT (66.8) and +6.3 over TrackFormer (63.3).
- On SportsMOT, SambaMOTR reaches 69.8 HOTA and 59.4 AssA, the best HOTA overall and +1.6 AssA over MeMOTR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SambaMOTR instead of MeMOTR | DanceTrack tracking-by-propagation · DanceTrack test | HOTA: 63.4 -> 67.2 | Helped |
| use SambaMOTR instead of OC-SORT | BFT tracking · BFT | HOTA: 66.8 -> 69.6 | Helped |
| use SambaMOTR instead of TransTrack | SportsMOT tracking · SportsMOT | HOTA: 68.9 -> 69.8 | Helped |
| add MaskObs to vanilla SSM query propagation (ablation) | SambaMOTR · DanceTrack test | HOTA: 63.5 -> 64.8; +1.3 HOTA | Helped |
| add synchronization across tracklets (ablation) | SambaMOTR with SSM and MaskObs · DanceTrack test | HOTA: 64.8 -> 65.9 | Helped |
| train on longer sequences (ablation) | SambaMOTR with SSM, MaskObs, and Sync · DanceTrack test | AssA: +1.9 | Helped |
| synchronize the fully updated state rather than not synchronizing (ablation) | SambaMOTR · DanceTrack test | HOTA: 66.0 -> 67.2 | Helped |
| add positional embeddings to observed queries (ablation) | SambaMOTR · DanceTrack test | HOTA: 65.6 -> 67.2 | Helped |
| predict a residual over the previous track query instead of predicting the full query (ablation) | SambaMOTR · DanceTrack test | HOTA: 64.2 -> 67.2 | Helped |
| propagate occluded queries with MaskObs instead of freezing the last observed state (ablation) | SambaMOTR · DanceTrack test | HOTA: 65.9 -> 67.2 | Helped |
| add synchronization across tracklets (ablation) | SambaMOTR · DanceTrack test hard sequences (average) | HOTA: 40.9 -> 44.5 | Helped |
Surprising findings
- On MOT17 SambaMOTR matches MeMOTR (58.8 HOTA) rather than clearly surpassing it, which the authors attribute to MOT17's linear motion and small size.
- Predicting a residual over track queries is far more effective than directly predicting the final track query (67.2 vs 64.2 HOTA).
Limitations
- Synchronization via self-attention gives quadratic complexity in the number of tracked sequences.
- Tracklets are dropped after N_miss inactive frames, so long disappearances (e.g. players leaving view for seconds) cause ID loss; no long-term re-identification.
- Evaluated only on Deformable DETR with ResNet-50 and on a few datasets; MOT17 gains over MeMOTR are negligible.
Experimental setup
Datasets: DanceTrack BFT SportsMOT MOT17 COCO
Benchmarks: DanceTrack BFT SportsMOT MOT17
Metrics: HOTA AssA DetA IDF1 MOTA
Baselines: MOTR MeMOTR TrackFormer TransTrack OC-SORT ByteTrack GTR C-BIoU QDTrack MOTRv2
Builds on: Mamba Deformable DETR MOTR MeMOTR TrackFormer
Closest papers by meaning
- TGFormer: Transformer with Track Query Group for Multi-Object Tracking (AAAI 2025 · similarity 0.88)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.87)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.86)
- DiffMOT: A Real-time Diffusion-based Multiple Object Tracker with Non-linear Prediction (CVPR 2024 · similarity 0.85)
- More Than Meets the Eye: Enhancing Multi-Object Tracking Even with Prolonged Occlusions (ICML 2025 · similarity 0.85)
- Tracktention: Leveraging Point Tracking to Attend Videos Faster and Better (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.