Research map / Adversarial robustness and security / Black-box and transfer attacks
Robust Adaptation of Foundation Models With Black-Box Visual Prompting
Changdae Oh, Gyeongdeok Seo, Geunyoung Jung, Zhi-Qi Cheng, Hosik Choi, Jiyoung Jung, Kyungwoo Song
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add input-dependent Coordinator prompts (BlackVIP) over VP w/ SPSA-GC (ablation) | VP w/ SPSA-GC · Biased MNIST (rho=0.8) | accuracy (%): 60.34 -> 66.21 | Improved |
| add input-dependent Coordinator prompts (BlackVIP) over VP w/ SPSA-GC (ablation) | VP w/ SPSA-GC · Loc-MNIST (1:1) | accuracy (%): 16.21 -> 69.08 | Improved |
| add input-dependent Coordinator prompts (BlackVIP) over VP w/ SPSA-GC (ablation) | VP w/ SPSA-GC · 14 few-shot benchmarks | avg accuracy (%): 58.8 -> 64.0 | Improved |
| add black-box prompting (BlackVIP) over zero-shot (ablation) | CLIP ViT-B/16 · 14 few-shot benchmarks | avg accuracy (%): 57.6 -> 64.0 | Improved |
| replace pretrained SSL encoder with PCA features (BlackVIP-SE) (ablation) | BlackVIP · 14 few-shot benchmarks | avg accuracy (%): 64.0 -> 63.5 | No clear effect |
| replace pretrained SSL encoder with PCA features (BlackVIP-SE) (ablation) | BlackVIP · Biased MNIST (rho=0.8) | accuracy (%): 66.21 -> 55.73 | Hurt |
| replace pretrained SSL encoder with PCA features (BlackVIP-SE) (ablation) | BlackVIP · ImageNet-1K train-time peak memory (ViT-B) | peak memory (MB): 2,428 -> 1,319 | Improved |
| replace pretrained SSL encoder with PCA features (BlackVIP-SE) (ablation) | BlackVIP · training runtime per batch | seconds: 0.2684 -> 0.1233 | Improved |
| add BlackVIP visual prompt to CuPL text prompt (ablation) | CuPL · EuroSAT | accuracy (%): 45.91 -> 68.88 | Improved |
| add BlackVIP visual prompt to LLM BBO text prompt (ablation) | LLM BBO · EuroSAT | accuracy (%): 52.80 -> 71.80 | Improved |
| add BlackVIP visual prompt to CuPL text prompt (ablation) | CuPL · DTD | accuracy (%): 55.85 -> 55.58 | No clear effect |
| increase Coordinator parameters from 9K to 150K in white-box regime (ablation) | Ours · SVHN/CLEVR/Pets average | avg accuracy (%): 64.6 -> 66.1 | Improved |
Closest papers by meaning
- Query Efficient Black-Box Visual Prompting with Subspace Learning (CVPR 2025 · similarity 0.92)
- Enhancing Visual Prompting through Expanded Transformation Space and Overfitting Mitigation (NeurIPS 2025 · similarity 0.89)
- LoR-VP: Low-Rank Visual Prompting for Efficient Vision Model Adaptation (ICLR 2025 · similarity 0.88)
- LaViP: Language-Grounded Visual Prompting (AAAI 2024 · similarity 0.88)
- SA²VP: Spatially Aligned-and-Adapted Visual Prompt (AAAI 2024 · similarity 0.88)
- Exploring the Transferability of Visual Prompting for Multimodal Large Language Models (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
