Research map / Adversarial robustness and security / Attacks on text and graphs

Your Scale Factors are My Weapon: Targeted Bit-Flip Attacks on Vision Transformers via Scale Factor Manipulation

CVPR 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Attacks and defenses in sequential settings · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Flip-S, a targeted bit-flip attack on quantized Vision Transformers that flips bits in quantization scale factors (not just weights) and uses a mutual exclusion strategy to keep flipped bits 4 KB apart, reaching high attack success with about 50 flips.

Problem

Targeted bit-flip attacks on quantized ViTs need hundreds of bit-flips and place flips closer than 4 KB, which RowHammer generally cannot realize in practice.

Approach

Flipping a bit in a quantization scale factor changes a whole group of dequantized weights, so few scale-factor flips can implant a trigger-activated Trojan; a gradient-based Scale-Factor-Search with weight-flip fallback and a mutual exclusion rule enforces 4 KB separation.

Key results

• On ImageNet 8-bit, Flip-S reaches ASR of at least 95.2% across five ViTs with 50 bit flips, while baselines are typically below 80.0%.
• Flip-S needs 8x-20x fewer bit flips than baselines for comparable ASR (e.g., 25 flips reach 91.2% ASR on DeiT-base 4-bit vs. 200-500 for baselines).
• Flip-S is the only method satisfying the 4 KB bit-flip separation requirement across all models.

Measured effects reported in the paper

Surprising findings

• Trigger-only (no bit flips) and weight-only flipping yield much lower ASR (roughly 32-38% and 69-97%), showing scale-factor flips are key.
• The OCM defense, which protects the last layer, fails to mitigate Flip-S (ASR 91.6-98.5%).

Limitations

• Evaluated only on image classification with ViT-family models and two datasets
• Assumes adversary can implement precise RowHammer bit flips at chosen locations
• Defense evaluation limited to OCM and prohibiting critical bits

Experimental setup

Datasets: CIFAR-10 ImageNet

Benchmarks: CIFAR-10 ImageNet

Metrics: attack success rate clean data accuracy 4KB bit-flip separation compliance

Baselines: TBT ProFlip HPT TrojViT OCM

Builds on: TBT ProFlip TrojViT attention rollout bitsandbytes

Closest papers by meaning

• Attention-Imperceptible Backdoor Attacks on Vision Transformers (AAAI 2025 · similarity 0.85)
• Boosting the Transferability of Adversarial Attack on Vision Transformer with Adaptive Token Tuning (NeurIPS 2024 · similarity 0.85)
• Bi-ViT: Pushing the Limit of Vision Transformer Quantization (AAAI 2024 · similarity 0.85)
• Crossfire: An Elastic Defense Framework for Graph Neural Networks Under Bit Flip Attacks (AAAI 2025 · similarity 0.84)
• SilentStriker: Toward Stealthy Bit-Flip Attacks on Large Language Models (NeurIPS 2025 · similarity 0.84)
• Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.84)