Research map / Adversarial robustness and security / Attacks on text and graphs

SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models

NeurIPS 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: LLM jailbreak and injection defense · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SilentStriker, a stealthy bit-flip attack on quantized LLMs that suppresses key output tokens plus a perplexity term so outputs stay fluent while task accuracy collapses.

Problem

Existing bit-flip attacks on LLMs either leave performance mostly intact (PrisonBreak) or degrade accuracy by producing gibberish with huge perplexity (GenBFA), making them easy to detect. The paper targets attacks that degrade task performance while keeping output natural.

Approach

A differentiable key-token loss that sums softmax probabilities of content (non-function) tokens from the original responses and squares the sum, combined with a non-negated perplexity loss to preserve fluency; bits are chosen per weight by largest numerical deviation, and modules are selected by progressive search.

Key results

• On INT8 LLaMA-3.1-8B-Instruct with 50 bit flips, GSM8K accuracy drops from 65.7% to 7.6% while GPT naturalness goes 66.0 to 63.0 and PPL rises to 60.4.
• GenBFA drives accuracy to 0% on all benchmarks but naturalness to 0 and PPL to about 5.5x10^5 on the same model.
• Across five models and both INT8 and FP4 settings, SilentStriker reduces accuracy to roughly 10% or below on most benchmarks, with 50 (INT8) or 100 (FP4) bit flips.

Measured effects reported in the paper

Surprising findings

• Naively flipping the highest-position bit in FP4 models fails to produce noticeable attack effects even after 500 flips; per-weight largest-deviation bit selection is needed.
• Removing the key-tokens loss leaves accuracy at 63.1%, showing minimizing perplexity alone does not degrade performance.

Limitations

• Evaluated only on dense models; MoE architectures with dynamic expert loading not studied.
• Stealthiness is judged by GPT-4o naturalness and PPL only, with no human evaluation or detection-defense testing.
• Attack assumes white-box access and RowHammer capability on edge devices without ECC.

Experimental setup

Datasets: GPT-4o generated attack questions DROP GSM8K TriviaQA-Wiki WikiText

Benchmarks: DROP GSM8K TriviaQA-Wiki WikiText

Metrics: accuracy (F1 for DROP, exact match for GSM8K and TriviaQA) GPT-4o naturalness score (0-100) perplexity

Baselines: PrisonBreak GenBFA

Builds on: GenBFA PrisonBreak spaCy

Closest papers by meaning

• Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.87)
• Stealth edits to large language models (NeurIPS 2024 · similarity 0.86)
• Better Language Model Inversion by Compactly Representing Next-Token Distributions (NeurIPS 2025 · similarity 0.86)
• Why Jailbreaks Succeed in Diffusion Language Models: An Energy Landscape Analysis (NeurIPS 2026 · similarity 0.86)
• Many-shot Jailbreaking (NeurIPS 2024 · similarity 0.86)
• Certifying Language Model Robustness with Fuzzed Randomized Smoothing: An Efficient Defense Against Backdoor Attacks (ICLR 2025 · similarity 0.86)