Research map / Federated learning and privacy / Privacy attacks and secure learning
Private Model Compression via Knowledge Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use adaptive rather than preset loss norm bound (ablation) | RONA · CIFAR-10 | testing accuracy | Helped |
| Add self-learning on public labels between teacher-distillation stages (ablation) | RONA · CIFAR-10 | training speed | Helped |
| Use the proposed query sample selection instead of random, margin, or diverse selection (ablation) | RONA · CIFAR-10 | student accuracy | Helped |
| Increase hint-learning epochs (ablation) | RONA · CIFAR-10 | student accuracy | Helped |
| Increase total distillation-learning epochs (ablation) | RONA · CIFAR-10 | student accuracy | Helped |
| Increase teacher-query batch size (ablation) | RONA · CIFAR-10 | student accuracy | Hurt |
| Use a moderate amount of Gaussian noise (ablation) | RONA · CIFAR-10 | student accuracy | Helped |
| Increase the number of teacher-query samples (ablation) | RONA · CIFAR-10 | student accuracy | Helped |
| Use only 20% of public samples as teacher queries (ablation) | RONA · CIFAR-10 | student performance | Helped |
| Add teacher knowledge transfer for classes absent from student training data (ablation) | RONA · MNIST: unseen digits 6 and 9 | accuracy: 0.00 -> 9.59; 46.75; 53.43 | Helped |
Closest papers by meaning
- DP-MemArc: Differential Privacy Transfer Learning for Memory Efficient Language Models (AAAI 2025 · similarity 0.84)
- Practical Privacy-Preserving MLaaS: When Compressive Sensing Meets Generative Networks (AAAI 2024 · similarity 0.84)
- Differentially Private Sum-Product Networks (ICML 2024 · similarity 0.84)
- United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.84)
- RAPID: Retrieval Augmented Training of Differentially Private Diffusion Models (ICLR 2025 · similarity 0.84)
- Delving into Differentially Private Transformer (ICML 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.