Research map / Vision-language and multimodal / Human and embodied modeling
MambaVLT: Time-Evolving Multimodal State Space Model for Vision-Language Tracking
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MambaVLT, a Mamba-based vision-language tracker whose time-evolving state-space memory transmits long-term multimodal target information and adaptively updates language and visual references.
Problem
Vision-language tracking with box, language, or both references struggles to exploit long-term temporal information and to adaptively update multimodal reference features; discrete context-prompt updating causes error accumulation.
Approach
A time-evolving hybrid multimodal state space block carries the final SSM state across frames as memory, combined with modality-guided bidirectional scans (text-first and template-first), a selective locality enhancement block, and a modality-selection module that weights visual vs. language references.
Key results
- On TNL2K with NL&BBOX reference, MambaVLT reaches 66.5 AUC versus 63.1 for UVLTrack-B, a 3.4-point gain.
- On MGIT with BBOX reference, MambaVLT reaches 65.7 AUC and 51.6 precision, surpassing GRM's 50.0 precision.
- Ablation on TNL2K shows the time-evolving hybrid state space adds 1.2, 1.5 and 1.9 AUC points for BBOX, NL and NL&BBOX tasks respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add time-evolving hybrid state space (ablation) | MambaVLT · TNL2K BBOX, NL, and NL&BBOX | AUC: 60.9 -> 62.1; 55.3 -> 56.8; 62.6 -> 64.5 | Helped |
| add modality-guided bidirectional scan (ablation) | MambaVLT with THSS · TNL2K BBOX, NL, and NL&BBOX | AUC: 62.1 -> 62.5; 56.8 -> 57.3; 64.5 -> 65.3 | Helped |
| add selective locality enhancement (ablation) | MambaVLT with THSS and MgB · TNL2K BBOX, NL, and NL&BBOX | AUC: 62.5 -> 63.1; 57.3 -> 57.5; 65.3 -> 65.7 | Helped |
| add modality-selection module (ablation) | MambaVLT with THSS, MgB, and SLE · TNL2K BBOX, NL, and NL&BBOX | AUC: 63.1 -> 63.3; 57.5 -> 58.4; 65.7 -> 66.5 | Helped |
| use MambaVLT instead of UVLTrack-B | NL&BBOX tracking · TNL2K | AUC: 63.1 -> 66.5 | Helped |
| use MambaVLT instead of MMTrack | NL&BBOX tracking · LaSOT | AUC: 70.0 -> 66.6 | Hurt |
| use MambaVLT instead of DecoupleTNL | NL&BBOX tracking · OTB99 | AUC: 73.8 -> 72.2 | Hurt |
Surprising findings
- On LaSOT and OTB99 the method underperforms some baselines in the language-only setting, which the authors attribute to ambiguous textual descriptions.
Limitations
- Underperforms some trackers on LaSOT, and in the language-only setting on LaSOT and OTB99.
- Relies on ablations of Mamba-based components with no reported efficiency measurements.
Experimental setup
Datasets: OTB99 LaSOT TNL2K MGIT RefCOCOg-google GOT-10k
Benchmarks: TNL2K LaSOT OTB99 MGIT
Metrics: AUC Precision Normalized Precision
Baselines: UVLTrack-B OSTrack-256 GRM QueryNLT JointNLT MMTrack DecoupleTNL SNLT
Builds on: Mamba VMamba OSTrack UVLTrack
Closest papers by meaning
- Dynamic Updates for Language Adaptation in Visual-Language Tracking (CVPR 2025 · similarity 0.87)
- MemVLT: Vision-Language Tracking with Adaptive Memory-based Prompts (NeurIPS 2024 · similarity 0.85)
- ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model (NeurIPS 2024 · similarity 0.84)
- Unifying Visual and Vision-Language Tracking via Contrastive Learning (AAAI 2024 · similarity 0.83)
- MMT-Bench: A Comprehensive Multimodal Benchmark for Evaluating Large Vision-Language Models Towards Multitask AGI (ICML 2024 · similarity 0.83)
- MVLM: Template-Free Tracking via Vision-Language Margin Confidence and Memory-Gated Tracking (CVPR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.