Research map / Adversarial robustness and security / Attacks on text and graphs
Attack for Defense: Adversarial Agents for Point Prompt Optimization Empowering Segment Anything Model
Xueyu Liu, Xiaoyi Zhang, Meilin Liu, Guangze Shi, Jia Shen, Yujie Wang, Cai Zhao, Ziyuan He, Yongfei Wu, Mingqiang Wei, Yongle Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply trained attack agent to ideal prompts (ablation) | SAM with ideal prompts · PASCAL VOC | mDSC: 78.5 -> 34.2 | Hurt |
| Apply defense agent after the attack (ablation) | SAM with attacked prompts · PASCAL VOC | mDSC: 34.2 -> 83.5 | Improved |
| Apply defense agent after the attack (ablation) | SAM with attacked prompts · Kvasir | mDSC: 59.7 -> 76.4 | Improved |
| Add PPD defense agent to feature-matching prompts | SAM with feature matching · PASCAL VOC | mDSC: 41.3 -> 73.0 | Improved |
| Add PPD defense agent to feature-matching prompts | SAM with feature matching · ISIC | mDSC: 66.4 -> 76.3 | Improved |
| Add PPD defense agent to feature-matching prompts | SAM with feature matching · Kvasir | mDSC: 31.4 -> 64.1 | Improved |
| Use adversarial PPD instead of PPO prompt optimization | feature-matching prompts on SAM · Kvasir | mDSC: 42.9 -> 64.1 | Improved |
| Use adversarial PPD instead of PPO prompt optimization | feature-matching prompts on SAM · PASCAL VOC | mDSC: 61.7 -> 73.0 | Improved |
| Use learned prompt encoder trained on natural images (VRP-SAM) on medical data | VRP-SAM · ISIC / Kvasir | mDSC: 58.2 (VOC) -> 5.9 (ISIC), 0.0 (Kvasir) | Hurt |
| Add PPD refinement on top of Matcher-style prompts vs running PPD | runtime · One-shot workflow time | prompt engineering time (s): 2.768 (FM-PPO) -> 1.249 (FM-PPD) | Improved |
Closest papers by meaning
- Plug-and-Play PPO: An Adaptive Point Prompt Optimizer Making SAM Greater (CVPR 2025 · similarity 0.88)
- Robust SAM: On the Adversarial Robustness of Vision Foundation Models (AAAI 2025 · similarity 0.86)
- Vanish into Thin Air: Cross-prompt Universal Adversarial Attacks for SAM2 (NeurIPS 2025 · similarity 0.85)
- TAPT: Test-Time Adversarial Prompt Tuning for Robust Inference in Vision-Language Models (CVPR 2025 · similarity 0.85)
- Phase and Amplitude-aware Prompting for Enhancing Adversarial Robustness (ICML 2025 · similarity 0.84)
- AAKR: Adversarial Attack-based Knowledge Retention for Continual Semantic Segmentation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
