Research map / Vision-language and multimodal / Cross-modal learning
Exploring Historical Information for RGBE Visual Tracking with Mamba
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MamTrack, a Mamba-based RGB-Event tracker with a gated Fusion Mamba module inserted into a Transformer encoder and a Mamba Historical Decoder with autoregressive queries, achieving SOTA on VisEvent, FELT, and FE108.
Problem
RGB-Event visual tracking suffers from complex, costly cross-modal fusion modules and ignores effective integration of historical temporal information about target appearance and motion.
Approach
Replace heavy cross-modal attention with Mamba-based gated fusion (Target-Aware Scan and Cross-Modality Scan) inserted into a Transformer encoder, and model target history with a Mamba-based historical decoder using learnable autoregressive queries, which offers linear complexity and parallel training.
Key results
- On VisEvent, MamTrack achieves 61.6% SR and 79.2% PR, beating AQATrack by 2.9% SR and SDSTrack by 1.9% SR.
- On FELT, MamTrack reaches 48.9% SR and 60.8% PR.
- On FE108, MamTrack achieves 66.4% SR / 94.2% PR, outperforming ViPT by 0.6% SR; on RGBT234 it beats ViPT by 0.7% SR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use both RGB and event inputs instead of event-only or RGB-only input (ablation) | MamTrack · VisEvent | SR: 40.5 -> 61.6; 59.0 -> 61.6 | Helped |
| Add Target-Aware Scan (TAS) (ablation) | MamTrack · VisEvent | SR: 60.7 -> 61.6 | Helped |
| Add Cross-Modality Scan (CMS) (ablation) | MamTrack · VisEvent | SR: 60.2 -> 61.6 | Helped |
| Add gate-based Cross-Fusion (CF) (ablation) | MamTrack · VisEvent | SR: 60.0 -> 61.6 | Helped |
| Add Feature Sequence Attention (FSA) (ablation) | MamTrack · VisEvent | SR: 60.6 -> 61.6 | Helped |
| Add Historical State Aware (HSA) (ablation) | MamTrack · VisEvent | SR: 60.4 -> 61.6 | Helped |
| Insert Fusion Mamba at layers 12, 15, and 18 rather than the other tested placements (ablation) | MamTrack · VisEvent | SR: 60.0; 61.0; 61.6; 61.1; 60.1; 60.6 | Mixed |
| Use a 3-layer Mamba historical decoder instead of no decoder or 1 or 6 Mamba layers (ablation) | MamTrack · VisEvent | SR: 59.9; 60.5; 61.6; 60.9 | Mixed |
| Use Mamba rather than Transformer in the 3-layer historical decoder (ablation) | MamTrack · VisEvent | SR: 61.0 -> 61.6 | Helped |
| Increase historical information length from 1 to 4 (ablation) | MamTrack · VisEvent | SR: 60.1 -> 60.4 -> 61.6 | Helped |
| Extend historical information length from 4 to 8 or 16 (ablation) | MamTrack · VisEvent | SR: 61.6 -> 59.8; 61.6 -> 59.7 | Hurt |
| Use MamTrack rather than ViPT | RGB-T tracking · LasHeR | SR: +1.7% | Helped |
Surprising findings
- Historical queries of length 8 and 16 reduce performance relative to length 4, attributed to limited sequences per iteration hurting generalization.
Limitations
- Does not exploit the high temporal resolution of event data, as acknowledged in the conclusion.
- Longer historical windows (8, 16) degrade performance, likely due to limited training sequences per iteration.
Experimental setup
Datasets: VisEvent FELT FE108 LasHeR
Benchmarks: VisEvent FELT FE108 LasHeR RGBT234
Metrics: Success Rate Precision Rate Overlap Precision
Baselines: AQATrack SDSTrack ViPT FENet MixFormer TransT HDETrack OSTrack STARK ATOM DiMP PrDiMP
Builds on: Mamba HiViT OSTrack AQATrack VMamba
Closest papers by meaning
- RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mamba (AAAI 2025 · similarity 0.89)
- Temporal Adaptive RGBT Tracking with Modality Prompt (AAAI 2024 · similarity 0.86)
- Single-Model and Any-Modality for Video Object Tracking (CVPR 2024 · similarity 0.85)
- Cross-modulated Attention Transformer for RGBT Tracking (AAAI 2025 · similarity 0.85)
- Collaborating Vision, Depth, and Thermal Signals for Multi-Modal Tracking: Dataset and Algorithm (NeurIPS 2025 · similarity 0.85)
- Revisiting motion information for RGB-Event tracking with MOT philosophy (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.