Research map / Detection and segmentation / Tracking
MMOT: The First Challenging Benchmark for Drone-based Multispectral Multi-Object Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces MMOT, a drone-based 8-band multispectral multi-object tracking benchmark with oriented bounding boxes, plus a Spectral 3D-Stem and orientation-aware adaptations of eight MOT trackers and YOLOv11 that show multispectral input beats RGB.
Problem
Drone-based multi-object tracking suffers from small targets, occlusion, and clutter where RGB appearance cues degrade, and no dedicated multispectral UAV MOT dataset exists.
Approach
Provide the first large-scale multispectral UAV MOT dataset with oriented bounding boxes and adapt existing MOT trackers via a Spectral 3D-Stem (RGB-weight-compatible 3D convolution with spectral folding), an orientation-aware Kalman filter, and an angle head for DETR-style trackers.
Key results
- BoT-SORT achieves the best MSI results on MMOT with 53.6 class-averaged HOTA and 46.2 MOTA.
- MSI input improves superclass-averaged HOTA over RGB by about 3.5 to 4.1 points for MOTR, MOTRv2 and MeMOTR (e.g., MOTR 38.16 to 41.64).
- Spectral 3D-Stem improves class-averaged HOTA over a 2D stem, e.g., MOTR 35.9 to 39.0 and MeMOTR 38.5 to 42.3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use MSI input instead of RGB | MOTR · MMOT superclass-averaged | HOTA: 38.16 -> 41.64 (+3.48) | Helped |
| use MSI input instead of RGB | MOTRv2 · MMOT superclass-averaged | HOTA: 44.48 -> 48.53 (+4.05) | Helped |
| use MSI input instead of RGB | MeMOTR · MMOT superclass-averaged | HOTA: 40.75 -> 44.69 (+3.94) | Helped |
| add Spectral 3D-Stem instead of using a 2D stem (ablation) | ByteTrack · MMOT MSI | HOTA: 40.3 -> 40.5 | Helped |
| add Spectral 3D-Stem instead of using a 2D stem (ablation) | BoT-SORT · MMOT MSI | HOTA: 52.8 -> 53.6 | Helped |
| add Spectral 3D-Stem instead of using a 2D stem (ablation) | MOTR · MMOT MSI | HOTA: 35.9 -> 39.0 (+3.1) | Helped |
| add Spectral 3D-Stem instead of using a 2D stem (ablation) | MeMOTR · MMOT MSI | HOTA: 38.5 -> 42.3 (+3.8) | Helped |
Surprising findings
- Total inter-frame IoU averages only 0.30 on MMOT versus about 0.9 in prior UAV MOT datasets, because of strong platform motion.
Limitations
- Annotating high-quality oriented bounding boxes requires substantial manual effort.
- Results on extremely small objects remain weak for all trackers.
Experimental setup
Datasets: MMOT
Benchmarks: MMOT
Metrics: HOTA MOTA IDF1 DetA AssA
Baselines: SORT ByteTrack OC-SORT BoT-SORT MOTR MOTRv2 MeMOTR MOTIP
Builds on: SORT ByteTrack BoT-SORT MOTR MOTRv2
Closest papers by meaning
- MUST: The First Dataset and Unified Framework for Multispectral UAV Single Object Tracking (CVPR 2025 · similarity 0.90)
- MM-Tracker: Motion Mamba for UAV-platform Multiple Object Tracking (AAAI 2025 · similarity 0.86)
- UAV3D: A Large-scale 3D Perception Benchmark for Unmanned Aerial Vehicles (NeurIPS 2024 · similarity 0.85)
- Multiview Aerial Visual RECognition (MAVREC): Can Multi-view Improve Aerial Visual Perception? (CVPR 2024 · similarity 0.85)
- Unleashing Multispectral Video's Potential in Semantic Segmentation: A Semi-supervised Viewpoint and New UAV-View Benchmark (NeurIPS 2024 · similarity 0.84)
- M$^2$E-UAV: A Benchmark and Analysis for Onboard Motion-on-Motion Event-Based Tiny UAV Detection (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.