Research map / Federated learning and privacy / Privacy attacks and secure learning
DCT-CryptoNets: Scaling Private Inference in the Frequency Domain
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DCT-CryptoNets, which runs quantization-aware-trained ResNets on JPEG-style low-frequency DCT inputs under TFHE (Concrete-ML) to cut ReLU and bootstrapping cost for private inference on ImageNet.
Problem
Private (FHE-encrypted) inference of deep CNNs is too slow and error-prone at ImageNet scale, because non-linear activations and homomorphic bootstrapping dominate latency and noise accumulation.
Approach
Feed JPEG-style 8x8 2D-DCT frequency tensors (luma-prioritized, sub-selected channels) into a ResNet with a stride-1 first conv and the first-layer ReLU removed, so spatial size and ReLU/PBS counts drop, and train with 4/5-bit quantization-aware training for TFHE encryption.
Key results
- On ImageNet, DCT-CryptoNets (64x56x56 DCT input, ResNet-18) reaches 66.3% top-1 with 8,562 s latency versus SHE's 45,000 s normalized to 96 threads (about 5.3x speedup).
- DCT input reduces PBS operations by about 30% and HOPs by 33.1% at 224x224 and 44.1% at 448x448 relative to RGB ResNet-18.
- Encrypted accuracy confidence interval narrows from ±2.5% (RGB) to ±1.0% (DCT) on ImageNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace RGB inputs with 64-channel DCT inputs | DCT-CryptoNets ResNet-18 · ImageNet | latency: 16,115 -> 8,562 | Helped |
| replace RGB inputs with 24×16² DCT inputs | ResNet-20 · CIFAR-10 | accuracy: 86.5% -> 90.5% | Helped |
| replace RGB inputs with 24×16² DCT inputs | ResNet-20 · CIFAR-10 | latency: 603 -> 565 | Helped |
| retain 64 low-frequency channels instead of RGB | ResNet-18 · ImageNet | accuracy: 66.1±2.5% -> 66.3±1.0% | No clear effect |
| use 4-bit rather than 5-bit quantization (ablation) | DCT-CryptoNets ResNet-18 · ImageNet | accuracy: 66.3% -> 62.9% | Hurt |
| use 4-bit rather than 5-bit quantization (ablation) | DCT-CryptoNets ResNet-18 · ImageNet | latency: 8,562 -> 6,961 | Helped |
| use 8-bit rather than 5-bit quantization (ablation) | DCT-CryptoNets ResNet-18 · ImageNet | accuracy: 66.3% -> 68.5% | Helped |
| use 8-bit rather than 5-bit quantization (ablation) | DCT-CryptoNets ResNet-18 · ImageNet | latency: 8,562 -> 41,998 | Hurt |
| add first-layer ReLU pruning (ablation) | DCT-Net · CIFAR-10 | accuracy: +0.0% -> -0.4% | Hurt |
| add first-layer ReLU pruning (ablation) | DCT-Net · ImageNet | accuracy: +2.2% -> +2.3% | Helped |
| add skip-connection ReLU pruning (ablation) | DCT-Net · CIFAR-10 | accuracy: +0.0% -> -2.67% | Hurt |
| add skip-connection ReLU pruning (ablation) | DCT-Net · HOPs | HOP reduction: -21.7% -> -24.3% | Hurt |
Surprising findings
- Retaining all 192 lossless DCT channels performs worse than 64 channels on mini-ImageNet and ImageNet, suggesting high-frequency components hinder learning.
- Pruning skip-connection ReLUs reduces ReLU count but increases HOPs in TFHE and costs accuracy, so only first-layer ReLU pruning is used.
Limitations
- Accuracy benefits of DCT are smaller on small images such as 32x32.
- Performance is highly sensitive to cryptographic hyperparameters (precision rounding, PBS error probability) and QAT bit-width, requiring manual tuning.
- ImageNet accuracy under encryption was estimated by subsampling (200-image subsets) rather than full validation via simulation, which is costly (~8 s per image).
Experimental setup
Datasets: CIFAR-10 mini-ImageNet Imagenette ImageNet
Benchmarks: CIFAR-10 mini-ImageNet Imagenette ImageNet
Metrics: top-1 accuracy latency (s) homomorphic operations (HOPs) 95% confidence interval of encrypted accuracy
Baselines: SHE DCT-Net Falcon CKKS ResNet-20 methods VGG-9 TFHE RGB ResNet-18
Builds on: Concrete-ML Brevitas SHE DCT-Net
Closest papers by meaning
- Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption (ICML 2024 · similarity 0.88)
- Jaguar: Fast Private CNN Inference with Power-of-Two Homomorphic Arithmetic (NeurIPS 2026 · similarity 0.87)
- PrivCirNet: Efficient Private Inference via Block Circulant Transformation (NeurIPS 2024 · similarity 0.87)
- United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.86)
- FicGCN: Unveiling the Homomorphic Encryption Efficiency from Irregular Graph Convolutional Networks (ICML 2025 · similarity 0.85)
- CipherPrune: Efficient and Scalable Private Transformer Inference (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.