Research map / Vision-language and multimodal / Cross-modal learning
Cross-Modal Stealth: A Coarse-to-Fine Attack Framework for RGB-T Tracker
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CMS, a two-stage coarse-to-fine adversarial patch attack (RGB StyleGAN2-latent patch plus thermal shape mask) with pixel- and semantic-level correlation-breaking losses and a point-tracking-based reprojection for physical deployment against RGB-T trackers.
Problem
Adversarial attacks on RGB-T cross-modal trackers are unexplored, and cross-modal complementarity makes attacks hard; physical patches also suffer distortion from viewing angles.
Approach
A coarse-to-fine latent-space generator produces an RGB patch (StyleGAN2-based semantic adversarial patch refined by a texture network) and a thermal black-mask shape from the same adversarial latent, trained with center-away, pixel-level and semantic-level correlation-breaking losses; a TAPIR keypoint-tracking homography reprojects the patch across frames for physical robustness.
Key results
- On RGBT234, CMS reduces ViPT precision rate by 0.257 and success rate by 0.249, outperforming CSA and TTAttack.
- On MANet, CMS drops PR by 0.428 and SR by 0.299 on RGBT234.
- In physical-domain tests on ViPT, PR and SR drop by 0.289 and 0.222 respectively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply CMS rather than TTAttack | RGBT tracking on RGBT234 · RGBT234, ViPT | PR: 0.695 -> 0.578 | Helped |
| apply CMS rather than CSA | RGBT tracking on LasHeR · LasHeR, ViPT | PR: 0.562 -> 0.310 | Helped |
| use CMS rather than CSA on MANet | MANet tracking on LasHeR · LasHeR, MANet | PR: 0.328 -> 0.389 | Hurt |
| use a two-stage coarse-to-fine patch rather than a single-stage patch (ablation) | CMS on ViPT · RGBT234 | PR: 0.648 -> 0.578; 0.621 -> 0.578 | Helped |
| add a thermal patch to the attack instead of using only the thermal patch (ablation) | CMS on ViPT · RGBT234 | PR: 0.782 -> 0.578 | Helped |
| add semantic-level correlation loss (ablation) | CMS on ViPT · RGBT234 | PR: 0.666 -> 0.578 | Helped |
| add pixel-level correlation loss (ablation) | CMS on ViPT · RGBT234 | PR: 0.604 -> 0.578 | Helped |
| use point tracking-based reprojection rather than center pasting (ablation) | CMS in physical domain · Physical domain, ViPT | PR: 0.851 -> 0.709 | Helped |
| use point tracking-based reprojection rather than random transformation (ablation) | CMS in physical domain · Physical domain, ViPT | PR: 0.801 -> 0.709 | Helped |
| increase patch size ratio from 1.5 to 1.6 (ablation) | CMS on ViPT · RGBT234 | PR: 0.600 -> 0.578 | Helped |
| increase patch size ratio beyond 1.6 (ablation) | CMS on ViPT · RGBT234 | PR: 0.578 -> 0.504; 0.504 -> 0.463; 0.463 -> 0.438 | Helped |
Surprising findings
- Random noise patches sometimes improve tracking of online trackers (APFNet, MANet) relative to clean conditions.
Limitations
- Physical evaluation is limited to a single tracker (ViPT) with a printed RGB patch and aluminum-foil thermal patch.
- Patch size ratio of 1.6 was chosen for physical feasibility, while larger ratios give stronger attacks.
Experimental setup
Datasets: RGBT234 LasHeR
Benchmarks: RGBT234 LasHeR
Metrics: precision rate success rate performance drop
Baselines: CSA TTAttack random noise patch
Builds on: StyleGAN2 TAPIR CSA NatPatch
Closest papers by meaning
- ACAttack: Adaptive Cross Attacking RGB-T Tracker via Multi-Modal Response Decoupling (CVPR 2025 · similarity 0.92)
- PURA: Parameter Update-Recovery Test-Time Adaption for RGB-T Tracking (CVPR 2025 · similarity 0.83)
- Cross-modulated Attention Transformer for RGBT Tracking (AAAI 2025 · similarity 0.81)
- Reproducibility Study on Adversarial Attacks Against Robust Transformer Trackers (NeurIPS 2024 · similarity 0.80)
- Temporal Adaptive RGBT Tracking with Modality Prompt (AAAI 2024 · similarity 0.80)
- A Dual Stealthy Backdoor: From Both Spatial and Frequency Perspectives (AAAI 2024 · similarity 0.76)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.