Research map / Learning theory and algorithms / Prediction and decision losses
Trustworthy Machine Learning through Data-Specific Indistinguishability
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Data-Specific Indistinguishability (DSI), a relaxation of input-independent indistinguishability, with an optimal multivariate Gaussian noise mechanism and a DP-SGD-like training framework to provide trust guarantees against memorization, backdoor poisoning, and copyright infringement.
Problem
Trust concepts such as memorization, data poisoning, and copyright can be modeled as constraints on the influence of data on a trained model, but input-independent indistinguishability (e.g., DP) gives heavy utility loss, especially in high dimensions.
Approach
Calibrate Gaussian noise to the specific output differences between the target training outcome and safe reference outcomes (leave-out subsets), via a convex dual over m Lagrange multipliers, so that only data-specific pairs are indistinguishable rather than all input pairs; noise is dimension-independent and works with black-box optimizers.
Key results
- On CIFAR-10 with ResNet-20, DSI-Local-SGD reaches 57.4% at epsilon=infinity... and 71.2% at epsilon=1 versus 55.3% for DP-SGD (Xiao et al. 2023b) at epsilon=1; WideResNet-16 gets 76.1% at epsilon=1 vs 56.8%.
- For GPT2-small fine-tuning, DSI at epsilon=2, delta=1e-5 keeps validation perplexity at 24.3 vs 20.7 without DSI for 1-duplicate setting, while reducing exposure from 2.2 to 0.8.
- Under backdoor attacks with m sources, DSI noise lowers attack success rate relative to no-DSI training, with the reduction growing as m increases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DSI-Local-SGD instead of DP-SGD | CIFAR-10 training with ResNet-20 and WideResNet-16 · CIFAR-10 test | accuracy | Helped |
| apply DSI guarantees during fine-tuning (ablation) | GPT2-small on WikiText-5 · WikiText-5 | exposure metric: 2.2 -> 0.8 | Helped |
| apply DSI guarantees during fine-tuning (ablation) | GPT2-small on WikiText-5 · WikiText-5 validation | perplexity: 20.7 -> 24.3 | Hurt |
| apply DSI guarantees during fine-tuning (ablation) | OPT-125M on WikiText-5 · WikiText-5 | exposure metric: 3.3 -> 0.6 | Helped |
| apply DSI guarantees during fine-tuning (ablation) | OPT-125M on WikiText-5 · WikiText-5 validation | perplexity: 20.1 -> 24.7 | Hurt |
| apply DSI guarantees during fine-tuning (ablation) | OPT-350M on WikiText-5 · WikiText-5 | exposure metric: 0.6 -> 0.3 | Helped |
| apply DSI guarantees during fine-tuning (ablation) | OPT-350M on WikiText-5 · WikiText-5 validation | perplexity: 16.1 -> 17.6 | Hurt |
| use DSI defense against Low-Frequency poisoning | PreAct-ResNet18 trained on poisoned CIFAR-10 · CIFAR-10 backdoor attack | attack success rate | Helped |
| increase the number of data sources (ablation) | DSI backdoor defense on CIFAR-10 · CIFAR-10 backdoor attack | attack success rate | Helped |
| increase the number of data sources (ablation) | DSI backdoor defense on CIFAR-10 · CIFAR-10 clean test | accuracy | Helped |
| increase the DSI parameter ε (ablation) | Stable Diffusion fine-tuned on paintings · copyright protection and style transfer | style-transfer fidelity | Helped |
| use a larger batch size (ablation) | DSI-Local-SGD · CIFAR-10 training | accuracy | Helped |
Surprising findings
- Empirical memorization rates are far below the theoretical DSI bounds (e.g., 2.05% empirical vs 12.3% bound for GPT2-small), indicating the guarantees are conservative.
Limitations
- Provable DSI bounds are conservative relative to empirical leakage.
- Analysis relies only on noise randomness and does not exploit subsampling amplification.
- Guarantees are one-sided in divergence and apply only to specified reference sets.
Experimental setup
Datasets: WikiText-103 CIFAR-10 poisoned CIFAR-10 (Low-Frequency and Blended attacks) artist paintings (425 images, 10 artists)
Benchmarks: CIFAR-10 WikiText-103
Metrics: test accuracy validation perplexity exposure exact memorization rate attack success rate epsilon-delta privacy budget
Baselines: DP-SGD Xiao et al. 2023b DP-SGD variant Yu et al. 2021 public-data DP Bao et al. 2023 mixup DP
Builds on: DP-SGD Renyi differential privacy Gaussian mechanism PAC privacy
Closest papers by meaning
- The Mainstays of Trustworthy Machine Learning (AAAI 2025 · similarity 0.85)
- U-trustworthy Models. Reliability, Competence, and Confidence in Decision-Making (AAAI 2024 · similarity 0.85)
- Provable Robustness against Backdoor Attacks via the Primal-Dual Perspective on Differential Privacy (NeurIPS 2026 · similarity 0.84)
- Towards Trustworthy Deep Learning (AAAI 2024 · similarity 0.84)
- Towards Trustworthy Federated Learning with Untrusted Participants (ICML 2025 · similarity 0.84)
- Certified Unlearning for Neural Networks (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.