Research map / Adversarial robustness and security / Certified and empirical robustness

On the Vulnerability of Adversarially Trained Models Against Two-faced Attacks

ICLR 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Robustness-accuracy trade-off analysis

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that adversarially trained models can be exploited by two-faced attacks that create a false sense of robustness during verification while remaining vulnerable at test time.

Problem

Adversarial training is used to improve adversarial robustness, but it may give a false sense of robustness; the paper studies whether adversarially trained models can be manipulated to appear robust during verification while being vulnerable later.

Approach

Introduces two-faced attacks: slight input-feature perturbations crafted so that the model appears robust in the verification stage but is adversarially vulnerable in deployment, exposing a weakness of adversarial training pipelines.

Surprising findings

• Adversarially trained models can be made to show a false sense of robustness in verification while remaining vulnerable to attacks.

Experimental setup

Builds on: adversarial training

Closest papers by meaning

• Robust Models are less Over-Confident (NeurIPS 2022 · similarity 0.92)
• Adversarial Training and Robustness for Multiple Perturbations (NeurIPS 2019 · similarity 0.92)
• Towards Deep Learning Models Resistant to Adversarial Attacks (ICLR 2018 · similarity 0.91)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.91)
• Certified Adversarial Robustness with Additive Noise (NeurIPS 2019 · similarity 0.91)
• On the Scalability of Certified Adversarial Robustness with Generated Data (NeurIPS 2024 · similarity 0.89)