Research map / Adversarial robustness and security / Attacks on text and graphs
PACO: Stabilizing Vision Embeddings along Local Paths for Robust Vision-Language Models
Qihang Tang, Jiacheng Pi, Zhiguo Yang, Xu Liu, Perley Xu, Wenjie Ruan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Captioning avg, APGD-Ensemble 2/255, LLaVA-7B | CIDEr: 44.0 -> 50.3 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · VQA avg, APGD-Ensemble 2/255, LLaVA-7B | accuracy: 28.4 -> 30.1 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Captioning avg, APGD-Ensemble 4/255, LLaVA-7B | CIDEr: 30.5 -> 35.3 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · VQA avg, APGD-Ensemble 4/255, LLaVA-7B | accuracy: 19.5 -> 20.9 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Targeted attack 8/255, avg CIDEr | CIDEr: 70.8 -> 80.1 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Targeted attack 8/255, success rate | attack success rate: 3.2% -> 0% | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Zero-shot classification avg, clean | accuracy: 58.7 -> 61.1 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Zero-shot classification avg, 2/255 | accuracy: 41.7 -> 45.8 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Zero-shot classification avg, 4/255 | accuracy: 28.9 -> 31.7 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · COCO image-to-text retrieval | Recall@1: 5.3 -> 7.1 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · Flickr30K image-to-text retrieval | Recall@1: 15.4 -> 18.6 | Improved |
| replace Sim-CLIP fine-tuning with PACO | PACO vs Sim-CLIP · POPE mean F1 | F1: 76.9 -> 77.8 | Improved |
Closest papers by meaning
- Robust CLIP: Unsupervised Adversarial Fine-Tuning of Vision Embeddings for Robust Large Vision-Language Models (ICML 2024 · similarity 0.89)
- SLADE: Shielding against Dual Exploits in Large Vision-Language Models (CVPR 2025 · similarity 0.89)
- Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions (NeurIPS 2025 · similarity 0.88)
- LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders (NeurIPS 2025 · similarity 0.87)
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.87)
- Enhancing CLIP Robustness via Cross-Modality Alignment (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
