Research map / Architectures and efficiency / RNNs and normalization
Learning to Linearize Deep Neural Networks for Secure and Efficient Private Inference
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use sensitivity-driven rather than even layer-wise ReLU allocation (ablation) | SENet on ResNet18 · CIFAR-100 | accuracy: 70.12 -> 75.88 | Helped |
| Add stage 2 ReLU mask identification instead of random mask placement (ablation) | SENet on ResNet18 · CIFAR-100 | accuracy: 59.12 -> 70.59 | Helped |
| Add stage 3 activation-similarity fine-tuning (ablation) | SENet on ResNet18 · CIFAR-100 | accuracy: 64.10 -> 70.59 | Helped |
| Add the PRAM loss during final fine-tuning (ablation) | SENet on ResNet18 · CIFAR-10 | accuracy: 94.12 -> 94.91 | Helped |
| Use channel-level rather than pixel-level ReLU masks (ablation) | SENet on WRN22-8 · CIFAR-100 | accuracy: 79.81 -> 79.3 | Hurt |
| Increase the teacher influence through higher lambda and beta values (ablation) | SENet final-stage distillation · CIFAR-10 | accuracy | Helped |
| Add PRAM loss to reduced-width SENet++ sub-models (ablation) | SENet++ sub-models with dropout rate below 1.0 · CIFAR-10, CIFAR-100, Tiny-ImageNet | accuracy | No clear effect |
| Use SENet rather than DeepReDuce at comparable ReLU budgets | ResNet18 · Tiny-ImageNet | accuracy: 64.66 -> 64.96 | Helped |
Closest papers by meaning
- DeepReDuce: ReLU Reduction for Fast Private Inference (ICML 2021 · similarity 0.90)
- ReLU Strikes Back: Exploiting Activation Sparsity in Large Language Models (ICLR 2024 · similarity 0.85)
- Convex Approximation of Two-Layer ReLU Networks for Hidden State Differential Privacy (NeurIPS 2025 · similarity 0.85)
- Disparate Impact on Group Accuracy of Linearization for Private Inference (ICML 2024 · similarity 0.85)
- Compelling ReLU Networks to Exhibit Exponentially Many Linear Regions at Initialization and During Training (ICML 2025 · similarity 0.84)
- Most ReLU Networks Admit Identifiable Parameters (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.