Research map / Detection and segmentation / Tracking
S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2-Track, an end-to-end query-based camera 3D multi-object tracker with 2D-prompted query initialization, uncertainty-aware probabilistic attention decoder, and hierarchical query denoising, reaching 66.3% AMOTA on nuScenes test.
Problem
End-to-end query-based 3D multi-object tracking from multi-view cameras struggles with occlusions and small objects, and lacks systematic improvements across query initialization, propagation, and matching.
Approach
Decomposes end-to-end query-based 3D MOT into query initialization, propagation, and matching, and improves each: initial queries from predicted 2D boxes and depth, Gaussian (probabilistic) attention scores in the decoder, and denoising queries split into positive/negative/ignored tiers by 3D IoU thresholds.
Key results
- Achieves 66.3% AMOTA on nuScenes test (ViT-L, full resolution), surpassing prior best end-to-end Sparse4D-v3 by 8.9% AMOTA.
- On nuScenes val, S2-Track-F with ViT-L reaches 0.652 AMOTA vs PF-Track-F 0.479 with V2-99 backbone.
- Ablation on small-resolution V2-99: UPD adds 4.4% AMOTA, PQI 2.4%, HQD 2.9%, all three give 0.458 AMOTA vs 0.394 baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add UPD (ablation) | S2-Track baseline · nuScenes val, small-resolution | AMOTA: 0.394 -> 0.438 | Helped |
| add PQI (ablation) | S2-Track baseline · nuScenes val, small-resolution | AMOTA: 0.394 -> 0.418 | Helped |
| add HQD (ablation) | S2-Track baseline · nuScenes val, small-resolution | AMOTA: 0.394 -> 0.423 | Helped |
| remove noise thresholds from HQD (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.440 -> 0.458 | Hurt |
| replace hierarchical level selection in HQD with a single threshold (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.445 -> 0.458 | Hurt |
| set HQD lower bound threshold to 0.20 rather than 0.30 (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.450 -> 0.458 | Hurt |
| set HQD lower bound threshold to 0.40 rather than 0.30 (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.448 -> 0.458 | Hurt |
| set HQD upper bound threshold to 0.60 rather than 0.70 (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.436 -> 0.458 | Hurt |
| set HQD upper bound threshold to 0.65 rather than 0.70 (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.455 -> 0.458 | Hurt |
| use stride 8 rather than stride 16 in PQI (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.456 -> 0.458 | Hurt |
| use stride 32 rather than stride 16 in PQI (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.446 -> 0.458 | Hurt |
| replace DETR3D decoder with PETR (ablation) | S2-Track-S · nuScenes val, small-resolution | AMOTA: 0.452 -> 0.458 | Hurt |
Surprising findings
- The PQI and HQD modules, not designed for uncertainty, also reduce estimated predictive uncertainty as measured by MC dropout and ensembles.
Limitations
- Evaluated only on camera-only inputs; LiDAR-camera fusion left for future work.
- Uncertainty is approximated with MC dropout and ensembles because transformers lack native uncertainty estimates.
- Inference speed is lower than PF-Track-S (7.5 vs 9.2 FPS).
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes tracking val nuScenes tracking test nuScenes detection val nuScenes detection test
Metrics: AMOTA AMOTP RECALL MOTA MOTP IDS mAP NDS
Baselines: PF-Track Sparse4D-v3 DQTrack MUTR3D STAR-TRACK ADA-Track HSTrack QTrack
Builds on: PF-Track DETR3D DN-DETR YOLOX MUTR3D
Closest papers by meaning
- ADA-Track: End-to-End Multi-Camera 3D Multi-Object Tracking with Alternating Detection and Association (CVPR 2024 · similarity 0.89)
- M3SOT: Multi-Frame, Multi-Field, Multi-Space 3D Single Object Tracking (AAAI 2024 · similarity 0.89)
- Focusing on Tracks for Online Multi-Object Tracking (CVPR 2025 · similarity 0.88)
- TGFormer: Transformer with Track Query Group for Multi-Object Tracking (AAAI 2025 · similarity 0.87)
- DiffusionTrack: Diffusion Model for Multi-Object Tracking (AAAI 2024 · similarity 0.87)
- SynCL: A Synergistic Training Strategy with Instance-Aware Contrastive Learning for End-to-End Multi-Camera 3D Tracking (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.