Research map / Federated learning and privacy / Differential privacy
Enhancing Learning with Label Differential Privacy by Vector Approximation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes vector approximation for label-DP classification: each label is randomized into a K-dimensional bit vector via RAPPOR-style flipping and the model is trained with sigmoid/BCE to match it, so accuracy degrades only slowly as the number of classes grows.
Problem
Local label differential privacy for multi-class classification degrades sharply as the number of classes K grows, because scalar label randomized response must keep the true label among many candidates.
Approach
Instead of flipping a label to a single scalar, encode it as a one-hot-like random vector where each bit is independently flipped with an epsilon/2 privacy budget (RAPPOR-style), and train a multi-output sigmoid network with binary cross entropy so outputs approximate per-class conditional probabilities; the theory bounds excess risk by the approximation error relative to the gap between the top t
Key results
- On CIFAR-100 at epsilon=2, vector approximation reaches 64.4% accuracy versus 55.0% for ALIBI at epsilon=3, and 71.8% at epsilon=4 versus 65.0% for ALIBI.
- On MNIST at epsilon=0.2, the method reaches 63.9% accuracy versus 58.0% for randomized response and 50.9% for ALIBI.
- On CIFAR-10 at epsilon=1, the method reaches 75.1% accuracy versus 71.0% for ALIBI, without pre-training.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use vector approximation instead of label flipping | Local label-DP classification with ResNet-18 · CIFAR-100 (ε=3) | accuracy: 55.0 -> 67.7 | Helped |
| Use vector approximation instead of label flipping | Local label-DP classification with ResNet-18 · CIFAR-100 (ε=4) | accuracy: 65.0 -> 71.8 | Helped |
| Use vector approximation instead of label flipping | Local label-DP classification with ResNet-18 · CIFAR-10 (ε=1) | accuracy: 67.6 -> 75.1 | Helped |
| Use vector approximation instead of label flipping | Local label-DP classification with ResNet-18 · CIFAR-10 (ε=4) | accuracy: 93.5 -> 93.2 | Hurt |
| Use vector approximation instead of label flipping | Local label-DP classification with a simple CNN · MNIST (ε=0.2) | accuracy: 58.0 -> 63.9 | Helped |
| Use vector approximation instead of label flipping | Local label-DP classification with a simple CNN · Fashion MNIST (ε=0.5) | accuracy: 60.1 -> 75.7 | Helped |
| Use vector approximation instead of label flipping | Local label-DP classification with a simple CNN · MNIST (ε=2.0) | accuracy: 97.7 -> 97.4 | Hurt |
| Use vector approximation as the number of classes K increases | Label-DP classification on synthesized data (σ=0.05) · Synthesized data, increasing K | accuracy | Mixed |
| Use vector approximation as the number of classes K increases | Label-DP classification on synthesized data (σ=2/K) · Synthesized data, increasing K | accuracy | Helped |
Surprising findings
- At large epsilon on MNIST (2.0), the method is slightly below randomized response and LP-2ST (97.4% vs 97.7%).
Limitations
- The privacy mechanism uses symmetric flipping probabilities and is not claimed to be optimal.
- Analysis is restricted to classification; regression is left for future work.
- Performance in the low-privacy regime (large epsilon) is not clearly better than existing methods.
Experimental setup
Datasets: MNIST Fashion MNIST CIFAR-10 CIFAR-100 synthetic Gaussian data on unit circle
Benchmarks: MNIST Fashion MNIST CIFAR-10 CIFAR-100 synthetic Gaussian data
Metrics: accuracy
Baselines: randomized response LP-2ST LP-1ST ALIBI RRWithPrior
Builds on: randomized response RRWithPrior ALIBI RAPPOR
Closest papers by meaning
- Retraining with Predicted Hard Labels Provably Increases Model Accuracy (ICML 2025 · similarity 0.87)
- Differential Privacy Under Class Imbalance: Methods and Empirical Insights (ICML 2025 · similarity 0.87)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.86)
- Multi-Class Support Vector Machine with Differential Privacy (NeurIPS 2025 · similarity 0.86)
- The Limits of Differential Privacy in Online Learning (NeurIPS 2024 · similarity 0.86)
- Differential Privacy in Scalable General Kernel Learning via $K$-means Nystr{\"o}m Random Features (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.