Research map / Large language models / Language, safety and interpretability

Best-of-N Jailbreaking

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Jailbreak attacks and defenses · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Best-of-N Jailbreaking, a black-box method that repeatedly samples randomly augmented versions of harmful prompts (text, image, audio) until a model produces a harmful response, showing power-law ASR scaling and gains from composition with prefix attacks.

Problem

Frontier AI systems, including closed-source LLMs, VLMs and ALMs, and defenses such as circuit breakers, remain vulnerable to jailbreaks; there is a need for simple, scalable, black-box red-teaming methods that work across input modalities.

Approach

Repeatedly sample random modality-specific augmentations (character scrambling, capitalization, noising for text; typographic image variations; speed/pitch/noise/volume changes for audio) of a harmful request until one elicits a harmful response, exploiting input variance and output stochasticity; ASR follows a power law in the number of samples N, which allows forecasting.

Key results

• BoN reaches 89% ASR on GPT-4o and 78% on Claude 3.5 Sonnet with N=10,000 text augmentations.
• BoN achieves 52% ASR against circuit-breaker Llama-3-8B-Instruct-RR and 67% against GraySwan Cygnet after system prompt tuning.
• Power-law fits on 1,000 samples forecast ASR at 10,000 samples with an average error of about 4.4% for text.

Measured effects reported in the paper

Surprising findings

• Resampling a successful BoN jailbreak prompt yields harmful output only about 30% of the time at temperature 1, so jailbreaks are not reliable individually.
• Going from temperature 1 to temperature 0 changes BoN ASR only slightly (e.g. 0.7% to 27.7% drop), suggesting augmentation variance drives success more than sampling entropy.

Limitations

• Classifier false positives (GPT-4o HarmBench grader) required manual review and filtering, and can cause premature termination.
• Individual successful jailbreaks are unreliable under resampling, and results rely on API models with nondeterminism.
• Augmentations were minimally optimized, and the power-law scaling is only verified up to about 10,000 samples.

Experimental setup

Datasets: HarmBench (159 standard direct requests) AdvBench (subset for many-shot prefix)

Benchmarks: HarmBench

Metrics: attack success rate sample efficiency forecasting error

Baselines: unaugmented requests resampling Circuit Breaking (Llama-3-8B-Instruct-RR) PAIR many-shot jailbreaking

Builds on: PAIR many-shot jailbreaking circuit breakers HarmBench

Closest papers by meaning

• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.91)
• JailPO: A Novel Black-Box Jailbreak Framework via Preference Optimization Against Aligned LLMs (AAAI 2025 · similarity 0.91)
• Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.90)
• VERA: Variational Inference Framework for Jailbreaking Large Language Models (NeurIPS 2025 · similarity 0.90)
• GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs (NeurIPS 2025 · similarity 0.90)
• AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models (ICLR 2024 · similarity 0.90)