Research map / Adversarial robustness and security / Certified and empirical robustness

Adversarially Robust Multi-task Representation Learning

NeurIPS 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Robustness-accuracy trade-off analysis · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Gives excess adversarial transfer-risk rates for multi-task representation learning under adversarial training, for Lipschitz and smooth nonnegative losses, plus new single-task adversarial Rademacher bounds via fat-shattering comparison.

Problem

Theoretical understanding of how multi-task representation learning on source tasks helps achieve adversarial robustness on a new target task with little data.

Approach

Analyzes two-stage adversarial MTRL (learn shared representation adversarially on source tasks, then fit target head) with a new notion of robust task diversity; reduces multi-task to single-task Rademacher complexity and uses a fat-shattering comparison inequality (Rudelson-Vershynin) to decouple function-class complexity from attack complexity, yielding ~sqrt(d) and sqrt(k) attack factors.

Key results

• For Lipschitz losses the excess adversarial transfer risk decays as Otilde(sqrt(d C(F)/n) + sqrt(d C(H)/(nt)) + sqrt(k C(F)/m)).
• For smooth nonnegative losses the bound is an optimistic (fast-rate when robust realizability holds) rate in m, nt and n.
• The adversarial overhead factor is Otilde(sqrt(d)) for source tasks and Otilde(sqrt(k)) for the target task under the bounded within-domain attack assumption.

Limitations

• Attacks are assumed to keep inputs within the data domain X, which may not hold for common attacks.
• The sqrt(d) dependence from the Gaussian chain rule may be an artifact, leaving a gap relative to linear-setting rates.
• Purely theoretical; no empirical experiments are reported.

Experimental setup

Metrics: excess adversarial transfer risk bounds Rademacher complexity bounds

Baselines: Dong et al. 2021 adversarial transfer linear analysis Awasthi et al. 2020 adversarial linear rates Yin et al. 2019 adversarial Rademacher bounds

Builds on: Tripuraneni et al. multi-task representation learning theory Awasthi et al. adversarial Rademacher analysis Yin et al. adversarial generalization Srebro et al. optimistic rates Rudelson-Vershynin covering inequality

Closest papers by meaning

• Adversarial Robustness in Multi-Task Learning: Promises and Illusions (AAAI 2022 · similarity 0.89)
• Adversarially robust transfer learning (ICLR 2020 · similarity 0.89)
• On Transfer of Adversarial Robustness from Pretraining to Downstream Tasks (NeurIPS 2023 · similarity 0.88)
• Adversarially Robust Hypothesis Transfer Learning (ICML 2024 · similarity 0.88)
• Enhancing Adversarial Transferability with Checkpoints of a Single Model's Training (CVPR 2025 · similarity 0.87)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.86)