Research map / Federated learning and privacy / Privacy attacks and secure learning
HEPrune: Fast Private Training of Deep Neural Networks With Encrypted Data Pruning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HEPrune, an FHE-based encrypted data pruning framework for private DNN/MLP training, using an HE-friendly score, client-aided masking, and ciphertext-wise pruning to speed up encrypted training.
Problem
FHE-based private training of neural networks is very slow because every encrypted sample must go through repeated forward and backward passes; existing plaintext data pruning methods need costly non-linear and sorting operations under encryption and leave sparse ciphertexts.
Approach
Computes an FHE-friendly importance score (L1 error between prediction and label, via two subtractions and a max), offloads threshold selection to the client via client-aided masking to avoid homomorphic sorting, and merges sparse ciphertexts (ciphertext-wise pruning) so pruning actually removes ciphertexts.
Key results
- Pruning ratio 0.9 gives about 6.6x end-to-end speedup across five datasets with accuracy drop as small as 0.25% on Face Mask Detection; on CIFAR-10 the full pipeline reaches 6.55x speedup with 96.31% vs 96.57% accuracy for HETAL.
- On CIFAR-10 with 1% of data, the paper reports roughly 16x speedup with about 0.6% accuracy drop.
- Training from scratch on MNIST with a 3-layer MLP at 1% data gives 60.8x speedup but a 5.26% accuracy drop; 40-50% data matches or slightly exceeds full-data accuracy.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply encrypted data pruning at p = 0.9 | HEPrune · MNIST | runtime (h): 276.75 -> 41.89 | Helped |
| Apply encrypted data pruning at p = 0.9 | HEPrune · CIFAR-10 | accuracy (%): 96.57±0.04 -> 96.31±0.06 | Hurt |
| Apply encrypted data pruning at p = 0.9 | HEPrune · CIFAR-10 | runtime (h): 293.3 -> 44.76 | Helped |
| Use naïve encrypted data pruning | HETAL on CIFAR-10 · CIFAR-10 | runtime (h): 293.3 -> 488.91 | Hurt |
| Add client-aided masking (ablation) | Prune Baseline on CIFAR-10 · CIFAR-10 | runtime (h): 488.91 -> 196.91 | Helped |
| Add HE-friendly score (ablation) | Client Aided on CIFAR-10 · CIFAR-10 | runtime (h): 196.91 -> 105.57 | Helped |
| Add HE-friendly score (ablation) | Client Aided on CIFAR-10 · CIFAR-10 | accuracy (%): 96.16±0.07 -> 96.31±0.06 | Helped |
| Add ciphertext-wise pruning (ablation) | HEFS on CIFAR-10 · CIFAR-10 | runtime (h): 105.57 -> 44.76 | Helped |
| Train with 1% of the data rather than the full dataset (ablation) | HEPrune · CIFAR-10 | accuracy drop (%): around 0.6% | Hurt |
| Train with 40% of the data rather than the full dataset (ablation) | HEPrune · CIFAR-10 | accuracy: without any loss of accuracy | No clear effect |
| Train with 1% of the data rather than the full dataset (ablation) | HEPrune, MLP from scratch · MNIST | accuracy (%): -5.26 | Hurt |
| Train with 40% of the data rather than the full dataset (ablation) | HEPrune, MLP from scratch · MNIST | accuracy (%): +0.03 | Helped |
Surprising findings
- Training on 40%-70% of CIFAR-10 yields slightly higher accuracy than full data.
- Naive plaintext pruning (prune baseline) increases total training time to 488.91 h versus 293.3 h for full-data HETAL.
Limitations
- Only simple MLP models were evaluated; CNNs and Transformers not covered.
- Experiments limited to relatively small datasets.
- Client participation is required for the masking step (client-aided), and meta-information like importance scores is sent to the client.
Experimental setup
Datasets: MNIST CIFAR-10 Face Mask Detection DermaMNIST SNIPS
Benchmarks: MNIST CIFAR-10 Face Mask Detection DermaMNIST SNIPS
Metrics: test accuracy end-to-end runtime (h) speedup
Baselines: HETAL Unencrypted training Prune Baseline FHESGD
Builds on: EL2N HETAL FHESGD CKKS dynamic data pruning
Closest papers by meaning
- CipherPrune: Efficient and Scalable Private Transformer Inference (ICLR 2025 · similarity 0.86)
- PrivCirNet: Efficient Private Inference via Block Circulant Transformation (NeurIPS 2024 · similarity 0.84)
- EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption (ICML 2025 · similarity 0.83)
- FicGCN: Unveiling the Homomorphic Encryption Efficiency from Irregular Graph Convolutional Networks (ICML 2025 · similarity 0.83)
- Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.83)
- DCT-CryptoNets: Scaling Private Inference in the Frequency Domain (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.