Research map / Adversarial robustness and security / Certified and empirical robustness
Adversarially Robust Hypothesis Transfer Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical study of adversarially robust hypothesis transfer learning, giving robust generalization and excess risk bounds for adversarial regularized ERM and proximal stochastic adversarial training that depend on the quality of auxiliary hypotheses.
Problem
Learning adversarially robust models when auxiliary hypotheses (expert knowledge or pre-trained models) are available as initialization, and understanding how their quality affects robust generalization.
Approach
Combines a target hypothesis class with a fixed weighted combination of auxiliary hypotheses under adversarial loss, and shows via stability and uniform-convergence style analysis (without convexity) that a good initialization yields fast O(1/n) robust generalization rates, otherwise O(1/sqrt(n)).
Key results
- A-RERM robust generalization bound is O(1/n) when auxiliary hypotheses have small robust loss, otherwise O(1/sqrt(n)).
- Robust excess risk bound achieves O(1/n) under near robust-realizability but O(1/n^{1/4}) in general.
- Proximal stochastic adversarial training (PSAT) gives a generalization bound scaling as roughly O(1/n + alpha) with the perturbation size alpha entering additively.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use an auxiliary initialization with small robust population loss | A-RERM · Robust generalization | robust generalization gap: O(1/n) | Helped |
| Use an auxiliary initialization that is not good (no small robust population loss) | A-RERM · Robust generalization | robust generalization gap: O(1/√n) | Hurt |
| Use an auxiliary initialization with small robust population loss | A-RERM · Robust excess risk | robust excess risk: ˜O(1/n) | Helped |
| Use an auxiliary initialization when near robust-realizability does not hold | A-RERM · Robust excess risk | robust excess risk: ˜O(1/n1/4) | Hurt |
| Initialize PSAT with an auxiliary model | PSAT · Robust generalization | generalization gap: O(1/n + α) | Helped |
| Use PSAT with perturbation budget α | PSAT · Robust generalization | generalization gap: O(1/n + α) | Hurt |
Limitations
- Analysis of PSAT has an additive term depending on perturbation size alpha that does not vanish with n.
- Excess risk rate degrades to O(1/n^{1/4}) without near robust-realizability.
- Assumes fixed auxiliary weights beta and bounded hypotheses/kernels; no experiments reported.
Experimental setup
Metrics: robust generalization gap robust excess risk
Builds on: Kuzborskij & Orabona hypothesis transfer learning Madry adversarial training Kuzborskij algorithmic stability bounds biased regularization
Closest papers by meaning
- Adversarially robust transfer learning (ICLR 2020 · similarity 0.89)
- On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks (NeurIPS 2023 · similarity 0.88)
- Adversarially Robust Multi-task Representation Learning (NeurIPS 2024 · similarity 0.88)
- Initialization Matters for Adversarial Transfer Learning (CVPR 2024 · similarity 0.88)
- Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.87)
- Provable Robust Overfitting Mitigation in Wasserstein Distributionally Robust Optimization (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.