Research map / Adversarial robustness and security / Attacks on text and graphs
Towards Poisoning Fair Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a clean-label data poisoning attack that degrades fairness of high-dimensional learned representations by maximizing mutual information between representation and sensitive attribute, using FLD score as a tractable proxy solved via gradient matching.
Problem
Fair representation learning (FRL) methods can be poisoned so that learned representations leak sensitive attributes and fairness degrades; existing poisoning attacks target scalar predictions and need labels, so they cannot attack representation-level fairness.
Approach
Attack the fairness of high-dimensional representations directly by maximizing I(z, a), which lower-bounds the attack effect on any downstream classifier's demographic parity; approximate the BCE-based objective with the Fisher LDA (FLD) separability score and craft poisons via elastic-net-regularized gradient matching.
Key results
- Proposed attack increased demographic parity violations in all 16 experimental settings (four victims on four datasets) while baselines often failed.
- FLD score tracks change in optimal BCE loss, supporting its use as a proxy (Appendix D.2).
- Attack had minimal impact on accuracy of predicting y from z in the reported experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed representation-level mutual-information attack | Fair representation learning models · Adult, German, COMPAS, and Drug Consumption | demographic parity violation | Helped |
| Use the proposed attack instead of existing baselines | Fair representation learning models · 16 experimental settings | demographic parity violation | Helped |
| Increase the poisoning-sample budget on the German dataset | The proposed attack · German | demographic parity violation | Mixed |
| Reduce batch size | The proposed gradient-matching attack · Fair representation learning | attack effectiveness | Hurt |
| Use the proposed attack | Fair representation learning models · Adult and German | BCE loss | Helped |
| Use the proposed attack | Fair representation learning models · Adult and German | label-prediction accuracy | No clear effect |
| Replace elastic-net regularization with L1-only regularization (ablation) | The proposed attack · Adult | perturbation L1 and L2 norms | Hurt |
| Replace elastic-net regularization with L1-only regularization (ablation) | The proposed attack · Adult | attack performance | No clear effect |
| Increase encoder depth, adversarial-classifier depth, and representation dimension to 128 (ablation) | CFAIR and CFAIR-EO · Adult | attack effectiveness | Helped |
Surprising findings
- Baseline attacks, when they do not fail, sometimes show severe decreases in attack performance as the poisoning budget grows.
- On the German dataset, DP violation exacerbation decreased with more poisoning samples in some runs, attributed to optimization instability.
Limitations
- Gaussianity assumption behind FLD optimality may not hold in general, so the proxy is approximate.
- Evaluated only on small tabular datasets with two-hidden-layer networks in the main experiments.
- Assumes full white-box knowledge of victim architecture, training and parameters.
Experimental setup
Datasets: Adult German COMPAS Drug Consumption
Benchmarks: Adult German COMPAS Drug Consumption
Metrics: BCE loss change demographic parity violation classification accuracy
Baselines: NRAA-a NRAA-y RAA-a RAA-y ENG-EUC ENG-sFLD
Builds on: Witches' Brew gradient matching Exacerbating algorithmic bias through fairness attacks Learning Fair Representations
Closest papers by meaning
- Fair Representation Learning with Controllable High Confidence Guarantees via Adversarial Inference (NeurIPS 2025 · similarity 0.87)
- Inherent Tradeoffs in Learning Fair Representations (NeurIPS 2019 · similarity 0.87)
- Can Private Machine Learning Be Fair? (AAAI 2025 · similarity 0.85)
- Adversarial Fairness Network (AAAI 2024 · similarity 0.85)
- AIM-Fair: Advancing Algorithmic Fairness via Selectively Fine-Tuning Biased Models with Contextual Synthetic Data (CVPR 2025 · similarity 0.84)
- On the Alignment between Fairness and Accuracy: from the Perspective of Adversarial Robustness (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.