Research map / Adversarial robustness and security / Certified and empirical robustness

Mitigating Feature Gap for Adversarial Robustness by Feature Disentanglement

AAAI 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AFD, an adversarial fine-tuning method that uses a linear feature disentangler to separate and remove "confused" latent features causing the natural-vs-adversarial feature gap, plus feature alignment with the naturally pre-trained model.

Problem

Adversarial fine-tuning (AFT) of naturally pre-trained models yields an increasing gap between last-hidden-layer features of natural and adversarial samples, limiting robustness.

Approach

Disentangle adversarial features into intrinsic and confused components via two linear blocks; train the confused-branch to predict the wrong top class, push intrinsic features away from confused features (angular distance), and align intrinsic features with frozen natural features of the pre-trained model. The disentangler folds into the final classifier at inference, so no extra modules are need

Key results

• On ResNet18, AFD reaches 61.66% average accuracy on CIFAR-10 and 52.70% PGD / 47.40% AA, the best robust accuracy among AFT methods, outperforming ARREST by 2.50% PGD and 2.68% AA.
• On WideResNet-34-10, AFD achieves the best average accuracy on all three datasets (e.g., 65.27 on CIFAR-10, 42.92 on CIFAR-100, 36.25 on Tiny-ImageNet), exceeding the second-best baselines by about 1.62% and 1.34% in robust and average accuracy.
• Ablation on ResNet18 CIFAR-10: adding the disentangler to vAFT raises PGD from 51.40 to 52.68; adding alignment raises it to 51.71; the full AFD reaches 52.70.

Measured effects reported in the paper

Surprising findings

• AFD does not achieve the best clean accuracy among AFT methods; the authors attribute this to only adversarial features being purified by the disentangler.
• The feature gap between natural and adversarial samples grows during adversarial fine-tuning for vanilla AFT, AFKD and ARREST, even though the best robustness is attained at the last epoch.

Limitations

• Experiments not conducted on ImageNet-1k or other large-resolution datasets due to compute.
• AFD gives lower clean accuracy than some AFT baselines (e.g., ARREST on CIFAR-100 and Tiny-ImageNet ResNet18).
• No adaptive attacks against the disentangler f1 were designed.

Experimental setup

Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet

Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet

Metrics: clean accuracy PGD-20 robust accuracy AutoAttack accuracy average accuracy training time

Baselines: vanilla AFT AFKD (AFT+RGKD) ARREST RiFT PGD-AT TRADES FreeAT FGSM-GA ATAS

Builds on: adversarial fine-tuning PGD-AT TRADES ARREST RGKD

Closest papers by meaning

• Adversarial Robustness through Disentangled Representations (AAAI 2021 · similarity 0.88)
• Achieving Robustness in the Wild via Adversarial Mixing With Disentangled Representations (CVPR 2020 · similarity 0.88)
• Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.87)
• Reducing Excessive Margin to Achieve a Better Accuracy vs. Robustness Trade-off (ICLR 2022 · similarity 0.86)
• AutoLoRa: An Automated Robust Fine-Tuning Framework (ICLR 2024 · similarity 0.86)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.86)