Research map / Federated learning and privacy / Privacy attacks and secure learning

PASS: Private Attributes Protection with Stochastic Data Substitution

ICML 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Encryption-based private learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes PASS, which protects private attributes by stochastically substituting each sample with another from a substitution dataset, using a cosine-similarity-based substitution distribution trained with an information-theoretic loss instead of adversarial training.

Problem

Removing private attributes (e.g., gender) from data while preserving useful attributes and general features; existing adversarial-training-based methods are vulnerable to stronger adversaries (the Probing Attack).

Approach

Replace adversarial training with stochastic data substitution: each input sample is replaced by a sample drawn from a substitution dataset with probability given by a softmax over cosine similarities of learned embeddings, trained with a fully differentiable, mini-batch-unbiased loss derived from an information-theoretic objective (mutual information with private, useful attributes and the input)

Key results

• On AudioMNIST, PASS reaches 0.0% NAG for gender and mNAG 55.0% versus -7.2% for the best-by-mNAG-worst baseline MaSS and 1.0% for ADV.
• On Motion Sense, PASS achieves 0.0% NAG for gender and ID with activity NAG 98.1%, giving mNAG 98.1% versus at most 47.1% for baselines (BDQ).
• On CelebA, PASS gets 4.9% NAG for Male and mNAG 72.9%, while the best baseline MSDA reaches 2.3% mNAG.

Measured effects reported in the paper

Surprising findings

• Existing adversarial-training methods show near-zero NAG against their own jointly trained classifiers but substantially higher NAG (e.g., gender 62-65% on Motion Sense) against a Probing Attack classifier.
• Several baselines (e.g., MaSS, PPDAR, MSDA on AudioMNIST) obtain negative mNAG, meaning they hurt useful attributes more than they protect privacy.

Limitations

• Theoretical bounds rely on assumptions such as deterministic attributes and finite categorical attributes; LDP interpretation holds only under certain assumptions.
• Substitution dataset is a random subset of training data (4096 points), so substitutes come from the existing data distribution.

Experimental setup

Datasets: AudioMNIST Motion Sense CelebA

Benchmarks: AudioMNIST Motion Sense CelebA

Metrics: Normalized Accuracy Gain (NAG) mean Normalized Accuracy Gain (mNAG)

Baselines: ADV GAP MSDA BDQ PPDAR MaSS SPAct Laplace Mechanism

Builds on: adversarial training based private attribute protection (GAP, MSDA, MaSS, PPDAR, privacy funnel information bottleneck SUIFT

Closest papers by meaning

• MaSS: Multi-attribute Selective Suppression for Utility-preserving Data Transformation from an Information-theoretic Perspective (ICML 2024 · similarity 0.88)
• Learning Robust and Privacy-Preserving Representations via Information Theory (AAAI 2025 · similarity 0.85)
• I Prefer Not to Say: Protecting User Consent in Models with Optional Personal Data (AAAI 2024 · similarity 0.84)
• DECO-Bench: Unified Benchmark for Decoupled Task-Agnostic Synthetic Data Release (NeurIPS 2024 · similarity 0.83)
• Perturb a Model, Not an Image: Towards Robust Privacy Protection via Anti-Personalized Diffusion Models (NeurIPS 2025 · similarity 0.83)
• PID: Prompt-Independent Data Protection Against Latent Diffusion Models (ICML 2024 · similarity 0.83)