Research map / Adversarial robustness and security / Attacks on text and graphs

Guaranteed Jailbreaking Defense via Disrupt-and-Rectify Smoothing

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: LLM jailbreak and injection defense · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DR-Smoothing, a two-stage (disrupt-then-rectify) smoothing defense against jailbreaks for black-box LLMs, with a generic theoretical bound on defense success probability.

Problem

Smoothing-based jailbreak defenses like SmoothLLM feed out-of-distribution perturbed prompts to the LLM, causing unpredictable behavior and over-refusal of benign prompts, and lack formal guarantees on defense success probability.

Approach

Adds a rectification stage (semantic operations such as paraphrase, spell-check, summarize, selected uniformly or by a learned policy) after random character/word disruption, analogous to denoised smoothing, and derives a Hoeffding-based bound linking per-trial refusal probability, number of trials N, and disruption strength q.

Key results

• On Vicuna, DR-Smoothing-Policy reduces GCG ASR from 95.2% (undefended) to 0% and PAIR ASR to 20%, while keeping IF accuracy at 36.7% versus 19.8% for SmoothLLM.
• On GPT-3.5-turbo against PAIR, DR-Smoothing-Policy achieves 12% ASR versus 24% for SemanticSmooth-Policy, with IF accuracy 61.2%.
• Under adaptive PAIR attacks, DR-Smoothing-Policy achieves ASR of 22% on Vicuna and 2% on Llama-2, whereas Erase-and-Check is lower on some models.

Surprising findings

• Erase-and-Check is strongest on harmlessness but collapses helpfulness (IF accuracy around 17-34%), illustrating the trade-off.
• Word-level disruptions are more effective against PAIR while character-level disruptions are more effective against GCG.

Limitations

• Adaptive attack evaluation limited to black-box PAIR since defense is non-differentiable; white-box adaptive attacks not evaluated.
• Requires N times more LLM queries than an undefended model.
• Theoretical bound relies on an assumed Lipschitz constant L for alpha(q) and assumes alpha(0)=0 without empirical estimation.

Experimental setup

Datasets: AdvBench InstructionFollow

Benchmarks: AdvBench InstructionFollow

Metrics: attack success rate instruction-level accuracy

Baselines: Perplexity Filter Erase-and-Check SmoothLLM SemanticSmooth-Uniform SemanticSmooth-Policy

Builds on: SmoothLLM SemanticSmooth Denoised Smoothing Randomized Smoothing Erase-and-Check

Closest papers by meaning

• Fight Back Against Jailbreaking via Prompt Adversarial Tuning (NeurIPS 2024 · similarity 0.91)
• Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks (NeurIPS 2024 · similarity 0.91)
• RobustKV: Defending Large Language Models against Jailbreak Attacks via KV Eviction (ICLR 2025 · similarity 0.90)
• JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.90)
• Systematic Scaling Analysis of Jailbreak Attacks in Large Language Models (NeurIPS 2026 · similarity 0.89)
• Understanding and Enhancing the Transferability of Jailbreaking Attacks (ICLR 2025 · similarity 0.89)