Research map / Adversarial robustness and security / Certified and empirical robustness
Improving Adversarial Robust Fairness via Anti-Bias Soft Label Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ABSLD, which adaptively re-temperates teacher soft labels per class based on student class-wise robust error gaps to improve adversarial robust fairness in adversarial robustness distillation.
Problem
Adversarial training and adversarial robustness distillation yield models with strong robustness on easy classes but weak robustness on hard classes (robust fairness problem).
Approach
Shows class-wise smoothness of soft labels affects robust fairness, and adjusts per-class teacher distillation temperatures (sharper for classes where the student has higher error risk) so the student's class-wise error gap shrinks; applies separate temperature sets for clean and adversarial examples.
Key results
- On CIFAR-10 ResNet-18 under PGD, ABSLD achieves worst-class robustness 36.50% and NSD 0.216, versus best baseline worst-class 33.30% (CFA).
- ABSLD reduces NSD by 0.028, 0.032, 0.017, and 0.024 vs the best baseline under FGSM, PGD, CW-inf, and AA on CIFAR-10 ResNet-18.
- On CIFAR-100 ResNet-18 under PGD, ABSLD achieves worst-10% robustness 10.50% and NSD 0.500, versus CFA at 8.40% and 0.563.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-10, FGSM | NSD: 0.230 -> 0.202 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-10, PGD | NSD: 0.258 -> 0.216 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-10, CW∞ | NSD: 0.288 -> 0.249 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-10, AA | NSD: 0.296 -> 0.256 | Helped |
| Apply ABSLD instead of the strongest baseline | MobileNet-v2 · CIFAR-10, PGD | NSD: 0.241 -> 0.224 | Helped |
| Apply ABSLD instead of the strongest baseline | MobileNet-v2 · CIFAR-10, AA | NSD: 0.331 -> 0.259 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-100, PGD | NSD: 0.563 -> 0.500 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-100, AA | NSD: 0.679 -> 0.601 | Helped |
| Apply ABSLD instead of the strongest baseline | ResNet-18 · CIFAR-10, PGD | worst-class robust accuracy: +3.2% | Helped |
| Apply ABSLD instead of the strongest baseline | MobileNet-v2 · CIFAR-10, PGD | worst-class robust accuracy: 35.00 -> 35.70 | Helped |
| Use sharper soft labels for hard classes and smoother soft labels for easy classes | SAT-trained ResNet-18 · CIFAR-10, class 4 | robust accuracy: 24.2% -> 30.2% | Helped |
| Use sharper soft labels for hard classes and smoother soft labels for easy classes | SAT-trained ResNet-18 · CIFAR-10, class 5 | robust accuracy: 30.9% -> 39.1% | Helped |
Surprising findings
- Average robustness is sometimes sacrificed: ABSLD's average clean and CW-inf robustness is below RSLAD/AdaAD on CIFAR-10 ResNet-18.
Limitations
- Theoretical analysis is restricted to a binary-class setting and extended to DNNs heuristically.
- Trade-off between average robustness and fairness remains; ABSLD sacrifices some average robustness on some attacks.
- Requires hyperparameters such as temperature bounds and temperature learning rate tuned per setting.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet
Benchmarks: CIFAR-10 CIFAR-100
Metrics: average robustness worst-class robustness normalized standard deviation
Baselines: SAT TRADES RSLAD AdaAD FRL BAT CFA Fair-ARD
Builds on: Adversarial Robustness Distillation RSLAD Fair-ARD TRADES Self-Adaptive Training
Closest papers by meaning
- DAFA: Distance-Aware Fair Adversarial Training (ICLR 2024 · similarity 0.90)
- Towards Fairness-Aware Adversarial Learning (CVPR 2024 · similarity 0.90)
- On the Alignment between Fairness and Accuracy: from the Perspective of Adversarial Robustness (ICML 2025 · similarity 0.89)
- Out of Thin Air: Exploring Data-Free Adversarial Robustness Distillation (AAAI 2024 · similarity 0.89)
- Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement (CVPR 2024 · similarity 0.89)
- Annealing Self-Distillation Rectification Improves Adversarial Training (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.