Research map / Adversarial robustness and security / Attacks on text and graphs
AGFT: Alignment-Guided Fine-Tuning for Zero-Shot Adversarial Robustness of Vision-Language Models
Yubo Cui, Xianchao Guan, Zijun Xiong, Zheng Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace hard-label supervision with calibrated soft pre-trained predictions (AGFT) vs TeCoA | CLIP ViT-B/32 adversarial fine-tuning · 15 zero-shot datasets, PGD-20 eps=1/255 | average robust accuracy (%): 38.51 -> 46.57 | Improved |
| replace hard-label supervision with AGFT | CLIP ViT-B/32 adversarial fine-tuning · 15 zero-shot datasets, clean | average clean accuracy (%): 56.93 -> 61.35 | Improved |
| use AGFT vs strongest baseline GLADIATOR | CLIP ViT-B/32 · 15 zero-shot datasets, PGD-20 eps=1/255 | average robust accuracy (%): 43.46 -> 46.57 | Improved |
| evaluate at larger inference budget 4/255 (fine-tuned at 1/255) | AGFT vs TGA-ZSR · PGD-20 eps=4/255 | average robust accuracy (%): AGFT 8.88 vs TGA-ZSR 10.30 | Hurt |
| fine-tune with larger budget eps=4/255 using AGFT | CLIP ViT-B/32 · 15 datasets | average clean accuracy (%): best baseline 41.73 -> 49.47 | Improved |
| use AGFT on RN50x4 backbone | CLIP RN50x4 · 15 datasets | average clean accuracy (%): clean 52.04 (TeCoA) vs 51.23; PGD robust 31.00 vs 34.69 | Mixed |
| use AGFT on ImageNet in-domain | CLIP ViT-B/32 · ImageNet PGD-20 | robust accuracy (%): TGA-ZSR 52.80 vs AGFT 44.95 | Hurt |
| use AGFT on distribution-shifted ImageNet-R | CLIP ViT-B/32 · ImageNet-R PGD-20 | robust accuracy (%): best baseline 41.08 -> 45.21 | Improved |
| decrease temperature scaling ratio gamma from 1.0 to 0.4 (calibration) (ablation) | AGFT · hyperparameter grid, 1/tau=180 | clean accuracy (%): 58.96 -> 61.35 | Improved |
| decrease gamma from 1.0 to 0.2 at 1/tau=100 (ablation) | AGFT · hyperparameter grid | robust accuracy (%): 45.07 -> 33.29 | Hurt |
| increase 1/tau from 100 to 220 at gamma=0.4 (ablation) | AGFT · hyperparameter grid | robust accuracy (%): robust 43.17 -> 47.08; clean 62.20 -> 60.68 | Mixed |
| add one extra forward pass for pre-trained distribution (vs PMG-AFT/TGA-ZSR) | AGFT training cost · training time per epoch | minutes: TGA-ZSR 39.78 -> AGFT 25.96 (TeCoA 23.45) | Improved |
Closest papers by meaning
- Improving Zero-Shot Adversarial Robustness in Vision-Language Models by Closed-form Alignment of Adversarial Path Simplices (ICML 2025 · similarity 0.93)
- On the Zero-shot Adversarial Robustness of Vision-Language Models: A Truly Zero-shot and Training-free Approach (CVPR 2025 · similarity 0.92)
- Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness (CVPR 2024 · similarity 0.90)
- Text-Guided Attention is All You Need for Zero-Shot Robustness in Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Few-Shot Adversarial Prompt Learning on Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Enhancing CLIP Robustness via Cross-Modality Alignment (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
