Research map / Vision-language and multimodal / Cross-modal learning
ACAttack: Adaptive Cross Attacking RGB-T Tracker via Multi-Modal Response Decoupling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ACAttack, an adaptive cross-attack framework that generates multi-modal adversarial patches (visible texture plus shared infrared shape) to fool RGB-T trackers in digital and physical settings.
Problem
Adversarial attacks on RGB-T multi-modal trackers have been largely unexplored; existing tracker attacks target RGB only and fail to break modal balance and cross-modal feature enhancement in RGB-T trackers.
Approach
Alternately weakens the modality with higher response contribution (via a reciprocal response-map distance softmax), adds a modal disturbance loss that pushes apart single-modal response maps, and a spatio-temporal loss with progressively enlarged pseudo-GT boxes; the patches share one adversarial shape for physical deployment.
Key results
- On RGBT234, ACAttack reduces ViPT PR from 0.835 to 0.621 and SR from 0.617 to 0.417.
- On RGBT234, ACAttack reduces SDSTrack PR from 0.848 to 0.616 and SR from 0.625 to 0.426.
- Patches trained on RGBT234 transfer to LasHeR, e.g., ViPT PR drops to 0.442 and SR to 0.350; physical-domain tests on 30 video sets show enlarged, mispositioned tracking boxes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply ACAttack instead of no attack | ViPT · RGBT234 | PR: 0.835 -> 0.621 | Helped |
| apply ACAttack instead of MTD | ViPT · RGBT234 | PR: 0.739 -> 0.621 | Helped |
| apply ACAttack instead of no attack | ViPT · LasHeR | SR: 0.525 -> 0.350 | Helped |
| apply ACAttack instead of no attack | BAT · RGBT234 | PR: 0.868 -> 0.750 | Helped |
| apply ACAttack instead of no attack | SDSTrack · LasHeR | SR: 0.531 -> 0.331 | Helped |
| add spatio-temporal joint loss (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.709 -> 0.621 | Helped |
| add modal disturbance loss (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.735 -> 0.621 | Helped |
| use K=0 coarse adversarial samples rather than the full setting (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.621 -> 0.672 | Hurt |
| use K=9 coarse adversarial samples rather than the full setting (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.621 -> 0.651 | Hurt |
| use alternating cross iteration rather than joint optimization (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.703 -> 0.645 | Helped |
| attack only RGB rather than both modalities (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.621 -> 0.691 | Hurt |
| attack only TIR rather than both modalities (ablation) | ACAttack on ViPT · RGBT234 | PR: 0.621 -> 0.669 | Hurt |
Surprising findings
- The RGB-only attack MTD caused a smaller performance drop than the proposed method on RGB-T trackers, and random noise alone produced only modest degradation.
Limitations
- Physical evaluation is limited to thermal and RGB patches on pedestrians captured by one UAV camera.
- Only three RGB-T trackers (ViPT, BAT, SDSTrack) are attacked.
Experimental setup
Datasets: RGBT234
Benchmarks: RGBT234 LasHeR
Metrics: precision rate success rate
Baselines: Random noise patch MTD ViPT BAT SDSTrack
Builds on: projected gradient descent MTD ViPT BAT SDSTrack
Closest papers by meaning
- Cross-Modal Stealth: A Coarse-to-Fine Attack Framework for RGB-T Tracker (AAAI 2025 · similarity 0.92)
- PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking (CVPR 2025 · similarity 0.88)
- Temporal Adaptive RGBT Tracking with Modality Prompt (AAAI 2024 · similarity 0.84)
- Cross-modulated Attention Transformer for RGBT Tracking (AAAI 2025 · similarity 0.84)
- RGBT Tracking via All-layer Multimodal Interactions with Progressive Fusion Mamba (AAAI 2025 · similarity 0.80)
- Revisiting motion information for RGB-Event tracking with MOT philosophy (NeurIPS 2024 · similarity 0.79)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.