Research map / Adversarial robustness and security / Certified and empirical robustness
Closing the Gap: Achieving Better Accuracy-Robustness Tradeoffs against Query-Based Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes activating inference-time defenses (RND, RCR, JPEG) only on low-confidence inputs, a training-free test-time method that improves accuracy-robustness tradeoffs against decision-based attacks PSJA and SurFree.
Problem
Existing defenses against query-based black-box attacks gain robustness only at a considerable cost in clean accuracy, since they perturb all inputs, benign ones included.
Approach
Query-based attacks must explore low-confidence regions, while genuine inputs are classified with high confidence; so a defense is enabled only when the classifier's max softmax confidence is below a threshold tau, and high-confidence inputs are processed normally. Applied to existing randomized/deterministic transformations without retraining, with temperature-scaling calibration.
Key results
- On CIFAR-10 against SurFree, the confidence-gated approach raises robust accuracy by up to 34% with almost no clean-accuracy loss.
- On CIFAR-10 against PSJA with RND, robust accuracy improves by up to 9% at a clean-accuracy cost of at most 2%, and on CIFAR-100 up to 20%.
- Against PSJA on CIFAR-10, training-time defenses (e.g., OAT, PNI) still outperform the training-free approach by at least 31% robust accuracy, although the method beats all other training-free defenses.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| activate RND only for low-confidence inputs instead of using the defense on all inputs | RND on CIFAR-10 against PSJA · CIFAR-10 | accuracy-robustness tradeoff: improves robust accuracy by 8% at CA=0.95; up to 9% with at most 1% decrease in CA | Helped |
| activate RCR only for low-confidence inputs instead of using the defense on all inputs | RCR on CIFAR-10 against PSJA · CIFAR-10 | CA: 67% to 94% | Helped |
| activate RND only for low-confidence inputs instead of using the defense on all inputs | RND on CIFAR-100 against PSJA · CIFAR-100 | RA: up to 8% | Helped |
| activate RCR only for low-confidence inputs instead of using the defense on all inputs | RCR on CIFAR-100 against PSJA · CIFAR-100 | CA: 19% | Helped |
| activate JPEG only for low-confidence inputs instead of using the defense on all inputs | JPEG on CIFAR-10 against SurFree · CIFAR-10 | RA: 34% | Helped |
| activate JPEG only for low-confidence inputs instead of using the defense on all inputs | JPEG on CIFAR-100 against SurFree · CIFAR-100 | RA: 21% | Helped |
| activate JPEG only for low-confidence inputs instead of using the defense on all inputs | JPEG on ImageNet against SurFree · ImageNet | RA: 21% | Helped |
| reduce JPEG quality parameter ν from 85 to 75 at τ=0.3 (ablation) | JPEG on ImageNet against SurFree · ImageNet | RA: 5% | Helped |
| vary confidence threshold τ from 1.0 to 0.8 with RND ν=0.05 (ablation) | RND on CIFAR-10 against PSJA · CIFAR-10 | CA: 82% to 94% | Helped |
| lower τ from 0.99 to 0.97 with RND ν=0.05 (ablation) | RND on CIFAR-10 against PSJA · CIFAR-10 | RA: 71% to 64% | Hurt |
| vary RND noise parameter ν while fixing τ=0.8 (ablation) | RND on CIFAR-10 against PSJA · CIFAR-10 | RA: 43% and 59% | Mixed |
| vary JPEG quality parameter ν while fixing τ=0.97 (ablation) | JPEG on CIFAR-10 against SurFree · CIFAR-10 | accuracy-robustness tradeoff: CA between 73% and 92%; RA between 71% and 83% | Helped |
Surprising findings
- Training-time defenses such as PNI reach much higher robust accuracy against PSJA than the proposed training-free method, showing a remaining gap.
- On ImageNet, robust accuracy decreases with larger noise and threshold, since noise can cause easier label flips that help the untargeted adversary.
Limitations
- Robust accuracy is evaluated on only 100 randomly selected correctly classified images per setting.
- Gains are limited against PSJA relative to training-time defenses; for ImageNet improvements are small (about 1%) for RND.
- Threshold tau must be tuned per defense and dataset, and the paper notes that the optimal tau varies with calibration.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNet
Metrics: clean accuracy robust accuracy attack success rate Pareto frontier
Baselines: PSJA with RND PSJA with RCR SurFree with JPEG OAT PNI RSE AT (RobustBench)
Builds on: PSJA SurFree HopSkipJump Random Self Ensemble temperature scaling
Closest papers by meaning
- Understanding the Robustness of Randomized Feature Defense Against Query-Based Adversarial Attacks (ICLR 2024 · similarity 0.87)
- Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.86)
- ReliabilityRAG: Effective and Provably Robust Defense for RAG-based Web-Search (NeurIPS 2025 · similarity 0.86)
- RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction (ICLR 2025 · similarity 0.86)
- Mind the Gap: Detecting Black-box Adversarial Attacks in the Making through Query Update Analysis (CVPR 2025 · similarity 0.85)
- Adapting to Evolving Adversaries with Regularized Continual Robust Training (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.