Research map / Adversarial robustness and security / Attacks on text and graphs

Sparse Fine-Tuning for Parameter-Efficient Adversarial Training

NeurIPS 2026 · Adversarial robustness and security › Attacks on text and graphs · approach: Adversarial prompts for language models

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes sparse fine-tuning as a parameter-efficient alternative to LoRA for adversarial training, aiming to reduce compute while closing the robustness gap.

Problem

Adversarial training is computationally expensive on large models and suffers from a clean-robust accuracy trade-off; existing LoRA-based parameter-efficient AT shows a notable robustness gap relative to full fine-tuning.

Approach

Applies sparse fine-tuning (updating a selected subset of weights) within adversarial training as a parameter-efficient method intended to avoid the robustness loss seen with LoRA-style low-rank adaptation.

Experimental setup

Metrics: clean accuracy robust accuracy

Baselines: LoRA-based adversarial training full fine-tuning adversarial training

Builds on: LoRA adversarial training

Closest papers by meaning

• Boosting Adversarial Robustness with CLAT: Criticality Leveraged Adversarial Training (ICML 2025 · similarity 0.90)
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.89)
• Pre-trained Model Guided Fine-Tuning for Zero-Shot Adversarial Robustness (CVPR 2024 · similarity 0.88)
• Initialization Matters for Adversarial Transfer Learning (CVPR 2024 · similarity 0.87)
• On the Duality Between Sharpness-Aware Minimization and Adversarial Training (ICML 2024 · similarity 0.87)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.87)