Research map / Adversarial robustness and security / Attacks on text and graphs
Adversarial Reinforced Instruction Attacker for Robust Vision-Language Navigation
Bingqian Lin, Yi Zhu, Yanxin Long, Xiaodan Liang, Qixiang Ye, Liang Lin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| attack instructions with DR-Attacker (ablation) | Base Agent · R2R Val Unseen | SR (%): 48.0 -> 46.6 | Hurt |
| attack instructions with DR-Attacker (ablation) | Base Agent · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 3.88 -> 1.95 | Hurt |
| add adversarial training with auxiliary task (ablation) | Base Agent · R2R Val Unseen | SR (%): 48.0 -> 49.6 | Improved |
| add finetuning on original data after adversarial training (ablation) | Adversarial Training w auxiliary task · R2R Val Unseen | SR (%): 49.6 -> 53.2 | Improved |
| add finetuning after adversarial training (ablation) | Adversarial Training w auxiliary task · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 3.90 -> 4.20 | Improved |
| add auxiliary self-supervised reasoning task (ablation) | Finetune (Adversarial Training) · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 4.08 -> 4.20 | Improved |
| add auxiliary self-supervised reasoning task (ablation) | Finetune (Adversarial Training) · CVDN Val Unseen (last answer, oracle) | Goal Progress (m): 3.37 -> 3.48 | Improved |
| use DR-Attacker instead of Static attacker in adversarial training (ablation) | Adversarial Training · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 3.97 -> 4.20 | Improved |
| use DR-Attacker instead of PWWS attacker in adversarial training (ablation) | Adversarial Training · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 3.09 -> 4.20 | Improved |
| use DR-Attacker instead of Heuristics attacker in direct attack (ablation) | Direct Attack · CVDN Val Unseen (last answer, mixed) | Goal Progress (m): 1.89 -> 1.95 | Mixed |
| use DR-Attacker instead of Static attack (ablation) | Direct Attack · CVDN Val Unseen (last answer, oracle) | Goal Progress (m): 1.71 -> 1.50 | Improved |
| compare with PREVALENT pretraining (large data, 8 V100) | DR-Attacker method · NDH training | Total time (min): 1661 -> 471 | Improved |
Closest papers by meaning
- Learning to Follow and Generate Instructions for Language-Capable Navigation (TPAMI 2023 · similarity 0.87)
- Steering Away from Harm: An Adaptive Approach to Defending Vision Language Model Against Jailbreaks (CVPR 2025 · similarity 0.86)
- Instruction Decomposition and Action Alignment for Vision-Language Navigation (ICML 2026 · similarity 0.86)
- Pandora's Box: Towards Building Universal Attackers against Real-World Large Vision-Language Models (NeurIPS 2024 · similarity 0.85)
- Everyday Object Meets Vision-and-Language Navigation Agent via Backdoor (NeurIPS 2024 · similarity 0.85)
- R-TPT: Improving Adversarial Robustness of Vision-Language Models through Test-Time Prompt Tuning (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
