Research map / Label-efficient and robust learning / Recognition and long-tail
The Fundamental Limits of Least-Privilege Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Formalises least-privilege learning (LPP) via maximal leakage conditioned on the task label, proves a fundamental utility-leakage trade-off under strictly positive posterior, and validates it empirically on LFWA+ (CNN256 with GRAD censoring), Texas Hospital (TabNet) and other settings.
Problem
Least-privilege learning aims to learn feature representations useful for a task while preventing inference of any unrelated sensitive attribute, but this goal has only been stated informally and its feasibility was unclear.
Approach
Formalise LPP as bounding the maximal leakage of a representation conditioned on the task label (so the task label's own fundamental leakage is allowed), and prove that under a strictly positive posterior any representation with utility above gamma must violate LPP with parameter gamma, matching the LDP trade-off.
Key results
- Theorem: under strictly positive posterior, LPP with parameter gamma and utility I(Y;Z) > gamma cannot hold simultaneously, so any useful representation leaks some other attribute beyond the task's fundamental leakage.
- On LFWA+ with CNN256, for every learning task there is at least one sensitive attribute with delta_Adv > 0, including under GRAD censoring.
- On Adult with MAX-ENT, the adversary's inference gain exceeds model utility for two of four sensitive attributes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use task-useful representations under strictly positive task-label uncertainty | Least-privilege learning · General data distributions with strictly positive posterior | utility–leakage trade-off | Hurt |
| Use a non-trivial feature map under strictly positive task-label uncertainty | Least-privilege learning · Perfect LPP (γ = 0) | conditional leakage | Hurt |
| Train with standard SGD rather than restricting representations to the task label | CNN256 · LFWA+ | attribute inference gain beyond fundamental leakage | Hurt |
| Apply gradient-reversal attribute censoring to the most-leaked attribute | CNN256 · LFWA+ | leakage of the censored attribute | Helped |
| Apply gradient-reversal attribute censoring to one sensitive attribute | CNN256 · LFWA+ | attribute inference gain beyond fundamental leakage | Hurt |
| Use MAX-ENT attribute obfuscation | Adult income model · Adult | attribute inference gain beyond utility gain: two out of four | Mixed |
| Share gradient updates instead of feature activations | Collaborative learning representations · Collaborative learning setting | attribute inference gain beyond utility gain | Hurt |
| Give the adversary direct access to feature representations rather than gradients | Collaborative learning representations · Collaborative learning setting | adversary inference gain | Helped |
| Use lower-layer rather than higher-layer representations | CNN256 · LFWA+ | adversary inference gain | Mixed |
| Change the encoder architecture from CNN256 to ResNet18 | Standard-SGD model · LFWA+ | attribute inference gain beyond utility gain | No clear effect |
| Use a TabNet encoder on Texas Hospital data | TabNet · Texas Hospital | attribute inference gain beyond utility gain | Hurt |
| Use a CNN256 encoder on LFWA+ | CNN256 · LFWA+ | attribute inference gain beyond fundamental leakage | Hurt |
Surprising findings
- Task labels can reveal sensitive attributes through fundamental leakage, e.g., 'Attractive' leaks gender via correlation, which is not obvious to data subjects.
- LDP implies LPP, so least-privilege learning offers no better utility-leakage trade-off than differential privacy.
Limitations
- Results hold only under the strictly positive posterior assumption on discrete finite spaces; other distributions may allow better trade-offs.
- Leakage measure is maximal leakage (worst-case over attributes in expectation), a specific operational measure; alternative measures may give different trade-offs.
- Empirical validation uses a limited set of learning techniques, architectures and datasets, and main experiments use one image dataset.
Experimental setup
Datasets: LFWA+ Adult Texas Hospital Discharge
Benchmarks: LFWA+ Adult Texas Hospital Discharge
Metrics: multiplicative gain (Arimoto information alpha=infinity) attribute inference gain delta_Adv = gain(S;Z|Y) - gain(Y;Z)
Baselines: GRAD censoring MAX-ENT adversarial representation learning standard ERM with SGD
Builds on: maximal leakage conditional entropy bottleneck local differential privacy gradient reversal
Closest papers by meaning
- Learning Robust and Privacy-Preserving Representations via Information Theory (AAAI 2025 · similarity 0.85)
- Guarantees for Nonlinear Representation Learning: Non-identical Covariates, Dependent Data, Fewer Samples (ICML 2024 · similarity 0.83)
- On the Provable Advantage of Unsupervised Pretraining (ICLR 2024 · similarity 0.83)
- MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation from an Information-theoretic Perspective (ICML 2024 · similarity 0.83)
- Obliviator Reveals the Cost of Nonlinear Guardedness in Concept Erasure (NeurIPS 2025 · similarity 0.83)
- High-dimensional Analysis of Knowledge Distillation: Weak-to-Strong Generalization and Scaling Laws (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.