Research map / Federated learning and privacy / Privacy attacks and secure learning

EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption

ICML 2025 · Federated learning and privacy › Privacy attacks and secure learning · approach: Encryption-based private learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Builds a GPU-accelerated OpenFHE/CKKS workflow and uses it to run an encrypted GPT-2 small forward pass with polynomial approximations of activations, reporting about 200x speedup over CPU.

Problem

Privacy-preserving LLM inference where a client's query must stay hidden from a cloud server, but FHE evaluation of transformers is too slow to be practical.

Approach

Provides a GPU-accelerated CKKS implementation integrated into OpenFHE and evaluates GPT-2 under FHE using low-degree polynomial approximations of GeLU, LayerNorm, and SoftMax (with a precomputed max lookup, batched evaluation), with accuracy checked on standard benchmarks.

Key results

• GPU-accelerated encrypted GPT-2 small forward pass (token position 128) is about 200x faster than the OpenFHE CPU baseline, reducing runtime from hours to minutes.
• Approximated GPT-2 small accuracy is close to the plaintext baseline across eight tasks (e.g., WiC 0.511 vs 0.492, HellaSwag 0.281 vs 0.289).
• Bootstrapping refreshes 20 levels in about 550 ms at 128-bit security and 45 levels in under 1 s at 80-bit security on an NVIDIA A100 80GB.

Measured effects reported in the paper

Surprising findings

• Approximated GPT-2 variants sometimes match or exceed baseline accuracy on some tasks, e.g., WiC for GPT-2 small improved from 0.492 to 0.511.

Limitations

• Only GPT-2 small is benchmarked under FHE; larger variants are only evaluated in plaintext approximation.
• Real-time chatbot latency remains out of reach; results target non-real-time tasks.
• No correctness guarantees against a malicious server, and clients may send arbitrary non-token vectors.

Experimental setup

Datasets: WebText

Benchmarks: HellaSwag ARC (Easy) PIQA Social IQa MNLI SST-2 ANLI WiC

Metrics: accuracy forward pass runtime

Baselines: OpenFHE CPU implementation unaltered GPT-2 plaintext baseline

Builds on: CKKS OpenFHE Cho et al. 2024 SoftMax optimization Zhang et al. 2024 CKKS transformer inference Dong et al. 2023 integer-only approximations

Closest papers by meaning

• Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.91)
• TensorCommitments: A Lightweight Verifiable Inference for Large Language Models (NeurIPS 2026 · similarity 0.87)
• An Efficient Private GPT Never Autoregressively Decodes (ICML 2025 · similarity 0.86)
• Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption (ICML 2024 · similarity 0.86)
• DR-Encoder: Encode Low-rank Gradients with Random Prior for Large Language Models Differentially Privately (AAAI 2025 · similarity 0.85)
• SentinelLMs: Encrypted Input Adaptation and Fine-Tuning of Language Models for Private and Secure Inference (AAAI 2024 · similarity 0.84)