Research map / Adversarial robustness and security / Attacks on text and graphs

Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SABER, a Beta-distribution-based scaling law that extrapolates Best-of-N jailbreak attack success rate (ASR@N) to large N from small per-query sampling budgets, with an anchored estimator that reduces MAE on HarmBench.

Problem

Predicting large-budget adversarial risk (ASR@N under Best-of-N sampling) for LLM jailbreaks from small measurement budgets, since exhaustive evaluation at large N is computationally infeasible.

Approach

Model per-query sample-level success probabilities as Beta(alpha, beta), derive the closed-form asymptotic law 1-ASR@N ~ Gamma(alpha+beta)/Gamma(beta) N^{-alpha}, fit (alpha, beta) by Beta-Binomial MLE, and anchor the extrapolation at an observed ASR@n to remove the unknown leading constant.

Key results

• With n=100 samples per query, the SABER-Anchored estimator predicts ASR@1000 with MAE 1.66 versus 12.04 for the naive baseline (86.2% error reduction across 12 attacker-victim-judge settings).
• For (Jailbreak-R1, GPT-4.1-mini, HarmBench Classifier) at n=10, SABER predicts ASR@50 within 0.25 points error versus 22.73 for the baseline.
• Predicted budget for 95% ASR is 465.9 vs ground truth 484 for Augmentation on Llama-3.1-8B with HarmBench Classifier.

Surprising findings

• Attack method rankings can reverse with sampling budget: Augmentation has lower ASR@1 than ADV-LLM on GPT-4.1-mini but surpasses it at N=15.
• Naive per-query SASR extrapolation strongly underestimates ASR@1000 in mid-range regimes (e.g., 63.40 predicted vs 75.16 ground truth for ADV-LLM on GPT-4.1-mini).

Limitations

• Scaling law fitted parameters are specific to the chosen data distribution, pipeline, and judge; extrapolation may not transfer when the distribution shifts.
• Small evaluation scale: only 159 HarmBench queries, one open and one closed victim model, and three attackers.
• Assumes i.i.d. attempts and a Beta mixture; the spike-and-slab extension for unbreakable queries is not empirically validated.

Experimental setup

Datasets: HarmBench

Benchmarks: HarmBench

Metrics: mean absolute error ASR@N relative error reduction

Baselines: Naive SASR-based BoN estimator Text Augmentation (best-of-n jailbreaking) ADV-LLM Jailbreak-R1

Builds on: Best-of-N Jailbreaking ADV-LLM Jailbreak-R1

Closest papers by meaning

• Scaling Trends in Language Model Robustness (ICML 2025 · similarity 0.91)
• Scaling Laws for Adversarial Attacks on Language Model Activations and Tokens (ICLR 2025 · similarity 0.89)
• REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective (ICML 2025 · similarity 0.89)
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.89)
• Confidence Elicitation: A New Attack Vector for Large Language Models (ICLR 2025 · similarity 0.88)
• Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.88)