Research map / Point clouds / Point cloud classification
M3SOT: Multi-Frame, Multi-Field, Multi-Space 3D Single Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes M3SOT, a lightweight transformer-based 3D single object tracker that combines multi-frame template sets, multi-receptive-field point features, and intermediate mask/center supervision, reaching state-of-the-art on KITTI, nuScenes, and WOD at 38 FPS.
Problem
3D single object tracking on sparse, occluded LiDAR point clouds, where existing methods underuse historical frames and surrounding context.
Approach
Many-to-one propagation of cues from multiple historical templates into the search area, combined with range-sampled multi-receptive-field DGCNN features and a hybrid transformer (GeoFormer, MaskFormer, SpaceFormer) with supervised intermediate mask and center predictions at each layer.
Key results
- Achieves 70.3 Success / 88.6 Precision mean on KITTI (M3SOT-F2 setting), outperforming CXTrack (67.5/85.3) and matching MBPTrack's success.
- On nuScenes, M3SOT-F1 reaches 30.6/33.7 mean, above the prior best CXTrack (26.5/31.5) and V2B.
- Runs at 38 FPS on a single RTX 3090 with a two-frame template set.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 2 template frames rather than 1 (ablation) | M3SOT · KITTI All | Success: 70.1 -> 70.0 | No clear effect |
| Use 2 template frames rather than 3 or 4 (ablation) | M3SOT · KITTI All | Success: 68.7 -> 70.0; 68.5 -> 70.0 | Helped |
| Use 2 template frames rather than 1 (ablation) | M3SOT · KITTI All | Precision: 88.7 -> 88.9 | Helped |
| Use feature ratios [2,4,8] instead of [1], [2], or [2,4] (ablation) | M3SOT · KITTI All | Success: 54.8 -> 70.0; 57.3 -> 70.0; 64.5 -> 70.0 | Helped |
| Add receptive-field ratio 16 to [2,4,8] (ablation) | M3SOT · KITTI All | Success: 70.0 -> 64.4 | Hurt |
| Add mask prediction task (ablation) | M3SOT with center task · KITTI All | Success: 70.1 -> 70.0 | No clear effect |
| Add center prediction task (ablation) | M3SOT with mask task · KITTI All | Success: 69.9 -> 70.0 | No clear effect |
| Add intermediate mask and center prediction tasks (ablation) | M3SOT without intermediate tasks · KITTI All | Success: 25.7 -> 70.0 | Helped |
| Replace range sampling with random sampling (ablation) | M3SOT · KITTI All | Success: 70.0 -> 66.0 | Hurt |
| Use many-to-one propagation instead of frame-wise self-attention (ablation) | M3SOT · KITTI Car | Precision: 71.6 -> 81.6 | Helped |
| Use many-to-one propagation instead of frame-wise cross-attention (ablation) | M3SOT · KITTI Car | Precision: 72.1 -> 81.6 | Helped |
| Use M3SOT rather than MBPTrack | 3D SOT trackers · Waymo Open Dataset Mean | Success: +0.6% | Helped |
Surprising findings
- Propagating cues frame-by-frame across consecutive template frames degrades precision (self-attention 82.1 to 71.6 with more frames), contrary to the intuition that temporal chaining helps.
- Using all points or half-sampled points as input performs poorly, while a sparse range-sampled three-layer EdgeConv setting works best.
Limitations
- Coordinating the integrated multi-component framework is difficult and configurations may vary across scenarios.
- Performance depends on template set size, which varies by category; more than 2 frames degrades results.
- Per-category results are not uniformly best; e.g., Pedestrian and Cyclist trail some baselines on certain metrics.
Experimental setup
Datasets: KITTI nuScenes Waymo Open Dataset
Benchmarks: KITTI nuScenes Waymo Open Dataset
Metrics: Success (IoU AUC) Precision (center distance AUC) FPS
Baselines: SC3D P2B LTTR MLVSNet BAT PTT V2B CMT PTTR STNet TAT M2-Track
Builds on: CXTrack P2B X-RPN DGCNN BAT
Closest papers by meaning
- Towards Category Unification of 3D Single Object Tracking on Point Clouds (ICLR 2024 · similarity 0.90)
- S2-Track: A Simple yet Strong Approach for End-to-End 3D Multi-Object Tracking (ICML 2025 · similarity 0.89)
- TrackCraft3R: Repurposing Video Diffusion Transformers for Dense 3D Tracking (NeurIPS 2026 · similarity 0.89)
- Modeling Continuous Motion for 3D Point Cloud Object Tracking (AAAI 2024 · similarity 0.88)
- Track-On: Transformer-based Online Point Tracking with Memory (ICLR 2025 · similarity 0.88)
- UAWTrack: Universal 3D Single Object Tracking in Adverse Weather (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.