Research map / Adversarial robustness and security / Attacks on text and graphs
On the Adversarial Robustness of Large Vision-Language Models under Visual Token Compression
Xinwei Zhang, Hangcheng Liu, Li Bai, Hao Wang, Qingqing Ye, Tianwei Zhang, Haibo Hu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add compression-aligned optimization (CAGE) over baseline attack | VEAttack on compressed LLaVA · TextVQA (K_model=64, avg) | robust accuracy (lower is stronger): 31.1 -> 18.9 | Improved |
| use CAGE instead of baseline attack | VEAttack on compressed LLaVA · VQA-v2 (K_model=16, avg) | robust accuracy (lower is stronger): 50.3 -> 42.9 | Improved |
| use CAGE instead of baseline attack | VEAttack on compressed LLaVA · GQA (K_model=192, avg) | robust accuracy (lower is stronger): 40.8 -> 36.2 | Improved |
| add the second objective (lambda) on top of EFD only (ablation) | CAGE · VQA-v2 (K_model=64) | robust accuracy: 45.8 -> 43.0 | Improved |
| add the second objective on top of EFD only (ablation) | CAGE · VQA-v2 (full tokens) | robust accuracy: 50.8 -> 49.4 | Improved |
| use attention score instead of l1-norm score for token importance (ablation) | CAGE · VQA-v2 (K_model=64) | robust accuracy: 60.6 -> 43.0 | Improved |
| use attention score instead of l1-norm score (ablation) | CAGE · VQA-v2 (K_model=16) | robust accuracy: 51.0 -> 42.7 | Improved |
| align attack budget with model token budget | VEAttack on LLaVA · K_model=16, K_attack 576 vs 16 | robust accuracy: 49.7 -> 44.4 | Improved |
| increase perturbation budget from 2/255 to 8/255 (ablation) | baseline attack · VQA-v2 (K_model=64) | robust accuracy: 53.5 -> 43.2 | Improved |
| increase perturbation budget from 2/255 to 8/255 (ablation) | CAGE · VQA-v2 (K_model=192) | robust accuracy: 46.5 -> 42.6 | No clear effect |
| apply D1 robustness-aware selection as defense | VisionZip · VQA-v2 (K_model=128) | robust accuracy under CAGE: 45.1 -> 50.0 | Improved |
| apply D1 defense | VisionZip · VQA-v2 (K_model=128) | clean accuracy: 72.1 -> 70.9 | Hurt |
Closest papers by meaning
- Variation-aware Vision Token Dropping for Faster Large Vision-Language Models (CVPR 2026 · similarity 0.90)
- SLADE: Shielding against Dual Exploits in Large Vision-Language Models (CVPR 2025 · similarity 0.89)
- Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models (ICML 2024 · similarity 0.89)
- Prune Redundancy, Preserve Essence: Vision Token Compression in VLMs via Synergistic Importance-Diversity (ICLR 2026 · similarity 0.89)
- ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.88)
- ApET: Approximation-Error Guided Token Compression for Efficient VLMs (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
