Research map / Adversarial robustness and security / Attacks on text and graphs

Learning Robust and Privacy-Preserving Representations via Information Theory

AAAI 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Membership inference and inversion · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ARPRL, an information-theoretic representation learning framework that jointly learns representations robust to adversarial examples and resistant to attribute inference, with utility preservation, and derives tradeoff and privacy-leakage bounds.

Problem

Machine learning models are vulnerable to both adversarial examples and private attribute inference; existing defenses address these separately and can even worsen privacy when robust.

Approach

Combines mutual-information objectives for adversarial robustness (representation vulnerability on worst-case perturbations), attribute privacy (vCLUB upper bound on I(z;u)), and utility (JS MI lower bound on I(x;z|u)) into one task-agnostic representation learner, and derives theorems on robustness/utility vs. privacy tradeoffs.

Key results

• On CelebA with alpha=0.9, robust accuracy 0.71, test accuracy 0.81 and gender inference accuracy 0.57 (gap 0.07), versus 0.87 / 0.91 / 0.81 at alpha=0.
• In the task-known privacy comparison on CelebA at equal test accuracy 0.88, ARPRL reaches 0.71 attribute inference accuracy versus 0.70 for Deepobfuscator and 0.79 for DPFE.
• Task-agnostic ARPRL obtains 0.87 robust accuracy on CelebA versus 0.89 for TRADES, rising to 0.91 when task labels are used.

Measured effects reported in the paper

Surprising findings

• Combining TRADES and Deepobfuscator for robustness and privacy gives a slightly worse tradeoff (0.79 robust, 0.84 test, 0.65 inference) than ARPRL, suggesting naive combination of SOTA methods is not the best option.
• Robust representations alone leak almost all private attribute information (near 100% inference accuracy on the toy example).

Limitations

• Mutual information neural estimators used are generally loose, so the bounds may not be tight.
• Privacy and robustness are controlled by hand-tuned alpha and beta, with a costly search.
• Attribute space is restricted to binary in the theory and attack evaluation, and the attacker is assumed not to access the representation learner.

Experimental setup

Datasets: CelebA Loans Adult Income synthetic two-circle toy dataset

Benchmarks: CelebA Loans Adult Income toy two-circle dataset

Metrics: robust accuracy (PGD) test accuracy attribute inference accuracy CW robust accuracy

Baselines: TRADES DPFE Deepobfuscator TRADES + Deepobfuscator

Builds on: representation vulnerability (Zhu et al. 2020) vCLUB MINE TRADES

Closest papers by meaning

• MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation from an Information-theoretic Perspective (ICML 2024 · similarity 0.87)
• Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.86)
• Unifying Re-Identification, Attribute Inference, and Data Reconstruction Risks in Differential Privacy (NeurIPS 2025 · similarity 0.85)
• A Unified Framework for Adversary-Aware Differential Privacy Bounds (NeurIPS 2026 · similarity 0.85)
• PASS: Private Attributes Protection with Stochastic Data Substitution (ICML 2025 · similarity 0.85)
• Task-Agnostic Privacy-Preserving Representation Learning for Federated Learning against Attribute Inference Attacks (AAAI 2024 · similarity 0.85)