Research map / Vision-language and multimodal / Cross-modal learning
Cross-Modality Perturbation Synergy Attack for Person Re-identification
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CMPS, a universal adversarial perturbation attack for cross-modality (visible-infrared) person re-ID that alternates gradients across modalities with a cross-modality triplet loss and random grayscale augmentation.
Problem
Security of cross-modality person re-identification models (visible vs infrared/thermal retrieval) against adversarial attacks has been largely ignored; existing ReID attacks target single-modality RGB matching and fail to exploit shared structure across modalities.
Approach
Learn a single universal perturbation by alternately updating it with gradients from visible and infrared modalities via a cross-modality triplet loss over identity cluster centroids, with random grayscale transformations as a bridge between modalities.
Key results
- On SYSU (visible to infrared, AGW baseline), the attack reduces rank-1 from 47.50% to 1.11%, versus 3.23% for the prior best (Col.+Del.).
- On RegDB (visible to thermal, AGW baseline), rank-1 drops from 70.05% to 2.29%, and the mAP is 3.92%.
- On LLCM (visible to infrared), rank-1 falls from 62.53% to 5.83%, compared with 8.61% for Col.+Del.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use CMPS instead of Col.+Del. | Cross-modality ReID attacks · SYSU, RegDB, and LLCM; both retrieval directions | rank-1 accuracy: 3.23 -> 1.11; 4.12 -> 1.31; 5.12 -> 2.29; 4.92 -> 1.93; 8.61 -> 5.83; 9.13 -> 6.42 | Helped |
| add CMPS to UAP-Retrieval (ablation) | UAP-Retrieval with AGW · RegDB and SYSU | mAP: 6.87 -> 3.98; 4.76 -> 3.42 | Helped |
| add CMPS to UAP-Retrieval (ablation) | UAP-Retrieval with AGW · RegDB and SYSU | rank-1: 5.53 -> 2.17; 5.09 -> 3.82 | Helped |
| add cross-modality augmentation to UAP-Retrieval (ablation) | UAP-Retrieval with AGW · RegDB and SYSU | mAP: 6.87 -> 5.11; 4.76 -> 3.85 | Helped |
| add cross-modality augmentation to CMPS (ablation) | CMPS with AGW · RegDB and SYSU | mAP: 3.98 -> 3.46; 3.42 -> 1.23 | Helped |
| vary adversarial bound epsilon from 2 to 4 to 8 to 16 (ablation) | CMPS with AGW · RegDB Visible to Thermal | rank-1 accuracy: 32.7% -> 9.6% -> 2.3% -> 0.3% | Helped |
| vary adversarial bound epsilon from 2 to 4 to 8 to 16 (ablation) | CMPS with AGW · RegDB Thermal to Visible | rank-1 accuracy: 40.5% -> 13.8% -> 2.0% -> 0.5% | Helped |
Surprising findings
- Classification attacks FGSM and PGD keep rank-1 above 60% on RegDB, performing worse than metric-based attacks, which the authors attribute to ReID relying on metric learning.
Limitations
- Transferability to unseen modalities and datasets is not established; the authors note the modalities encountered in practice may be unknown.
- Gradient-based methods may suffer catastrophic forgetting and inconsistency across models and modalities.
- The theoretical analysis relies on convexity assumptions that do not hold for deep networks.
Experimental setup
Datasets: SYSU-MM01 RegDB LLCM
Benchmarks: SYSU-MM01 RegDB LLCM
Metrics: Rank-1 Rank-10 Rank-20 mAP
Baselines: FGSM PGD Metric-FGSM LTA ODFA Col.+Del. UAP-Retrieval
Builds on: UAP-Retrieval Col.+Del. Metric-FGSM MI-FGSM
Closest papers by meaning
- Cross-Modality Person Re-identification with Memory-Based Contrastive Embedding (AAAI 2023 · similarity 0.91)
- Infrared-Visible Cross-Modal Person Re-Identification with an X Modality (AAAI 2020 · similarity 0.91)
- Joint Color-irrelevant Consistency Learning and Identity-aware Modality Adaptation for Visible-infrared Cross Modality Person Re-identification (AAAI 2021 · similarity 0.90)
- Empowering Visible-Infrared Person Re-Identification with Large Foundation Models (NeurIPS 2024 · similarity 0.87)
- Heterogeneous Test-Time Training for Multi-Modal Person Re-identification (AAAI 2024 · similarity 0.87)
- Shallow-Deep Collaborative Learning for Unsupervised Visible-Infrared Person Re-Identification (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.