Research map / Federated learning and privacy / Privacy attacks and secure learning
All Rivers Run to the Sea: Private Learning with Asymmetric Flows
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Delta, a private training and inference framework that splits IRs via SVD/DCT into a small private low-dimensional model and a residual part offloaded to a public GPU model after DP noise and binary quantization.
Problem
Protecting sensitive training and inference data in cloud ML when private environments (TEEs) are slow and cryptographic methods are too costly, while DP on raw inputs/IRs degrades utility.
Approach
Exploit asymmetric structure of intermediate representations (low-rank across channels, low-frequency across space): keep the low-dimensional information-sensitive part in a private small model and offload only the high-dimensional residuals, which need much less DP noise, to a public large model, with binary quantization and a split-logit private backpropagation.
Key results
- Under epsilon=1.4 on ResNet-18/ImageNet, Delta reaches 65.9% accuracy vs 34.4% for naive-DP perturbing IR, up to 31.5 points better.
- On ResNet-18/CIFAR-100 with batch 32, Delta training takes 62 ms per iteration (22x speedup over Priv-only 1372 ms) and inference 20 ms (25x over 510 ms).
- Delta achieves CIFAR-10 accuracy of 92.4% with noise on residuals, close to the original model, while naive-DP gets 69.6%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| perturb IRres instead of the full IR (ablation) | Delta · CIFAR-10 | accuracy: 69.6% -> 92.4% | Helped |
| perturb IRres instead of the full IR (ablation) | Delta · CIFAR-100 | accuracy: 48.3% -> 71.4% | Helped |
| perturb IRres instead of the full IR (ablation) | Delta · ImageNet | accuracy: 34.4% -> 65.9% | Helped |
| use asymmetric decomposition and residual offloading | ResNet-18 · CIFAR-100 training | time per iteration: 1372 -> 62 ms | Helped |
| use asymmetric decomposition and residual offloading | ResNet-18 · CIFAR-100 inference | time per iteration: 510 -> 20 ms | Helped |
| add DP noise to residuals (ablation) | Delta · CIFAR-100 model inversion | SSIM: 0.18 -> 0.09 | Hurt |
| add DP noise to residuals (ablation) | Delta · CIFAR-100 model inversion | AccM: 6.75% -> 2.13% | Hurt |
| add DP noise to residuals (ablation) | Delta · CIFAR-100 membership inference (5k public samples) | Acc: 0.56 -> 0.52 | Hurt |
| use DP with ε=1.4 instead of ε=1 (ablation) | Delta · CIFAR-100 membership inference (10k public samples) | Acc: 0.60 -> 0.55 | Hurt |
| remove residuals (ablation) | Delta · CIFAR-100 model inversion | SSIM: 0.08 -> 0.09 | Hurt |
Limitations
- Only semi-honest threat model; malicious parties not handled (verifiable computing suggested).
- Evaluated only on ResNet models and image classification; asymmetry for other architectures is shown only briefly (language models in appendix).
- Side-channel and denial-of-service attacks are out of scope.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 ImageNet
Benchmarks: CIFAR-10 CIFAR-100 ImageNet
Metrics: validation accuracy running time (ms) SSIM attack accuracy F1 MACs
Baselines: Slalom 3LegRace naive-DP (perturb IR) Priv-only Orig (non-private)
Builds on: 3LegRace Slalom SecretRevealer ML-Leaks
Closest papers by meaning
- Learning from End User Data with Shuffled Differential Privacy over Kernel Densities (ICLR 2025 · similarity 0.85)
- Certification for Differentially Private Prediction in Gradient-Based Training (ICML 2025 · similarity 0.85)
- United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.85)
- Differentially Private Prototypes for Imbalanced Transfer Learning (AAAI 2025 · similarity 0.85)
- Provable Privacy with Non-Private Pre-Processing (ICML 2024 · similarity 0.85)
- Position: Considerations for Differentially Private Learning with Large-Scale Public Pretraining (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.