Research map / Federated learning and privacy / Privacy attacks and secure learning

Privacy-Preserving Embedding via Look-up Table Evaluation with Fully Homomorphic Encryption

ICML 2024 · Federated learning and privacy › Privacy attacks and secure learning · approach: Encryption-based private learning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HELUT and CodedHELUT, CKKS-based encrypted look-up table evaluation built on a noise-cleansing encrypted indicator function, enabling encrypted word-embedding layers (GloVe, GPT-2, BERT) without bootstrapping.

Problem

Efficient homomorphic encryption evaluation of embedding layers (large look-up tables over token indices) in privacy-preserving NLP, where encrypting one-hot token vectors yields impractically large ciphertexts and runtimes.

Approach

An encrypted indicator function (EIF) built from recursive squaring (SqMethod) plus a noise-cleansing polynomial (Cleanse) yields near-exact one-hot vectors under CKKS; combining it with coded (compositional) input indices and compressed codebooks (Shu & Nakayama 2018) reduces multiplications from O(p^l) to O(p·l).

Key results

• CodedHELUT on synthetic 2^12-input LUT with p=8, l=2 with SIMD p1 runs in 3.83 ms total at 15-bit precision versus 81.16 ms for HELUT-LT (14 bits).
• Amortized evaluation time of 0.018-0.242 s for GloVe 6B50d embedding layer across configurations, with worst precision 16 bits in all settings.
• For GPT-2 and BERT embedding layers, encrypted evaluation takes 0.262-3.283 s amortized per input, with compression MSE around 0.001-0.013 for the compressed embedding.

Measured effects reported in the paper

Surprising findings

• Smaller p (8 vs 16) gave better HELUT cost and MSE at pl=512 because of lower CKKS bit consumption, despite more codebook lookups.

Limitations

• Relies on the compressed-embedding method of Shu & Nakayama (2018) which may limit generality
• Compression reduces accuracy: higher MSE and lower sentiment accuracy for small pl settings
• Cost still grows with embedding dimension d and with pl

Experimental setup

Datasets: GloVe 6B50d GloVe 42B300d GPT-2 embeddings BERT embeddings IMDB

Benchmarks: synthetic LUT (Z64 to R16) GloVe 6B50d GloVe 42B300d GPT-2 embedding layer BERT embedding layer IMDB sentiment analysis

Metrics: amortized running time (ms/s) precision (bits) MSE of compressed embedding sentiment classification accuracy

Baselines: HELUT-LT HELUT-CI

Builds on: CKKS compressed coded embedding (Shu & Nakayama, 2018) Badawi et al. 2020 encrypted one-hot LUT

Closest papers by meaning

• Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.87)
• EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption (ICML 2025 · similarity 0.84)
• Split-and-Denoise: Protect large language model inference with local differential privacy (ICML 2024 · similarity 0.84)
• United We Stand: Accelerating Privacy-Preserving Neural Inference by Conjunctive Optimization with Interleaved Nexus (AAAI 2024 · similarity 0.83)
• Synthesizing Privacy-Preserving Text Data via Finetuning *without* Finetuning Billion-Scale LLMs (ICML 2025 · similarity 0.83)
• FicGCN: Unveiling the Homomorphic Encryption Efficiency from Irregular Graph Convolutional Networks (ICML 2025 · similarity 0.83)