Research map / Vision-language and multimodal / Multimodal LLMs

MVLM: Template-Free Tracking via Vision-Language Margin Confidence and Memory-Gated Tracking

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Bias-aware multi-image robustness

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MVLM, a template-free, language-only visual tracker that gates between compact ROI search and global re-localization using a memory-smoothed vision-language margin confidence, with theoretical failure bounds.

Problem

Language-only (template-free) single object tracking is unstable over large search regions because VL correlation suffers spatial uncertainty and ambiguous saliency, and existing language trackers still rely on visual templates or initial boxes.

Approach

Combines a vision-language correlation margin, classification-head margin, and an EWMA temporal memory into a confidence score that decides whether to search a local ROI or re-localize globally, plus a correlation margin loss that suppresses hard background tokens; theory bounds failure into ROI exclusion and within-ROI mis-localization.

Key results

• MVLM reaches 73.0 PRE / 66.9 AUC on TNL2K and 80.4 PRE / 72.7 AUC on LaSOT in the tracking-by-bbox-and-language setting, with best TNL2K PRE among those compared.
• In the tracking-by-language setting, MVLM reports 60.9 PRE / 57.8 AUC on TNL2K, 65.5 PRE / 60.7 AUC on LaSOT, and 55.5 PRE on MGIT.
• Ablation shows local ROI search improves average PRE/AUC by 7.9%/6.8% over global search, and LCM plus MVLM add a further 0.8%/0.5%.

Surprising findings

• Adding the MVLM gate on top of LCM yields only a small average gain (0.8% PRE, 0.5% AUC), while the ROI search itself accounts for most of the improvement.

Limitations

• Theoretical bounds rely on assumptions such as sub-Gaussian token noise, equal ROI region sizes and a minimum margin, which may not hold in practice.
• Gains over the strongest baselines are modest on several benchmarks (e.g., LaSOT AUC 66.9 vs 66.6 type gaps, and MGIT AUC below MambaVLT).

Experimental setup

Datasets: TNL2K LaSOT VastTrack

Benchmarks: TNL2K LaSOT OTB99 MGIT

Metrics: PRE AUC NPR

Baselines: JointNLT QueryNLT MemVLT UVLTrack-B SUTrack-B224 MambaVLT DUTrack-256 TNL2K-1 CTRNLT

Builds on: CLIP HiViT SUTrack UVLTrack

Closest papers by meaning

• Towards More Flexible and Accurate Object Tracking With Natural Language: Algorithms and Benchmark (CVPR 2021 · similarity 0.88)
• MemVLT: Vision-Language Tracking with Adaptive Memory-based Prompts (NeurIPS 2024 · similarity 0.87)
• ChatTracker: Enhancing Visual Tracking Performance via Chatting with Multimodal Large Language Model (NeurIPS 2024 · similarity 0.87)
• MIMTrack: In-Context Tracking via Masked Image Modeling (AAAI 2025 · similarity 0.86)
• Unifying Visual and Vision-Language Tracking via Contrastive Learning (AAAI 2024 · similarity 0.86)
• Masked Diffusion Vision-Language Models for Temporal Action Localization (NeurIPS 2026 · similarity 0.86)