Research map / Architectures and efficiency / Attention and transformers
Nimbus: Secure and Efficient Two-Party Inference for Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Nimbus, a two-party secure Transformer inference framework combining a client-side outer-product HE protocol for linear layers with distribution-aware low-degree polynomial approximations of GELU and Softmax over smaller rings.
Problem
Secure two-party inference for Transformers is slow due to expensive HE-based matrix multiplications in linear layers and costly high-degree polynomial approximations of GELU and Softmax (exponential) in non-linear layers.
Approach
Shift the weight encryption to the client side (server sends encrypted weights in setup) so the client computes row-wise-encoded outer products with its plaintext activation shares, eliminating input ciphertext communication and yielding compact outputs; and fit low-degree piecewise polynomials for GELU/exp using the empirical input distribution, allowing smaller rings (Z_2^32, s=12) with a free u
Key results
- Nimbus improves end-to-end BERT inference by 2.7x-4.7x over Bumblebee across LAN and WAN settings, and linear-layer matmul by 7.2x-12.5x (LAN) and 2.9x-4.0x (WAN).
- Distribution-aware polynomial approximations of GELU and Softmax are 2.9x-4.0x faster than Bumblebee, with average GLUE accuracy loss of about 0.6% without finetuning and 0.08% with lightweight finetuning.
- Linear-layer communication is reduced to 11.51% of Iron's and about 65% of Bumblebee's on one Transformer block.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Nimbus distribution-aware polynomial approximation instead of Bumblebee's approximation | BERT · GLUE Avg. | average score: 83.11 -> 82.53 | Hurt |
| add lightweight fine-tuning to Nimbus (ablation) | Nimbus · GLUE Avg. | average score: 82.53 -> 83.04 | Helped |
| add lightweight fine-tuning to Nimbus (ablation) | Nimbus · GLUE tasks | average accuracy loss vs FP baseline: 0.6% -> 0.08% | Helped |
| replace Bumblebee with Nimbus's combined optimizations | BERT Transformer block · LAN and WAN | end-to-end speedup: 2.7× ~ 4.7× | Helped |
| use COP for linear layers instead of Bumblebee's protocol | BERT linear layers · LAN | speedup: 7.2× ~ 12.5× | Helped |
| use COP for linear layers instead of Bumblebee's protocol | BERT linear layers · WAN | speedup: 2.9× ~ 4.0× | Helped |
| use Nimbus's GELU approximation instead of Bumblebee's | BERT · GELU, LAN | speedup: 3.4× | Helped |
| use Nimbus's GELU approximation instead of Bumblebee's | BERT · GELU, WAN | speedup: 3.3× | Helped |
| use Nimbus's Softmax approximation instead of Bumblebee's | BERT · Softmax, LAN | speedup: 4.0× | Helped |
| use Nimbus's Softmax approximation instead of Bumblebee's | BERT · Softmax, WAN | speedup: 2.9× | Helped |
| use Nimbus instead of Bumblebee | BERT Softmax · one Transformer block | communication: 354.26 -> 115.35 megabytes | Helped |
| use Nimbus instead of Bumblebee | BERT GELU · one Transformer block | communication: 185.13 -> 53.22 megabytes | Helped |
Surprising findings
- Low-degree polynomials with a smaller ring Z_2^32 and precision s=12 avoid the overflow that causes Bumblebee's high-degree approximation to fail at precision above 10 bits.
Limitations
- Client must store encrypted weights on disk (ciphertexts at least four times larger than shares), adding memory and storage overhead.
- Without finetuning the approximation loses about 0.6% average GLUE accuracy; 0.08% requires lightweight finetuning.
- Activation-activation multiplications (QK^T and PV) and LayerNorm reuse Bumblebee's latency and are not optimized.
Experimental setup
Datasets: GLUE
Benchmarks: GLUE CoLA SST-2 MRPC STS-B QQP MNLI QNLI RTE
Metrics: accuracy Matthews correlation F1 Pearson correlation latency communication cost rounds
Baselines: Iron Bumblebee MPCFormer BOLT
Builds on: Cheetah Iron Bumblebee SecretFlow
Closest papers by meaning
- Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption (ICML 2024 · similarity 0.89)
- Ditto: Quantization-aware Secure Inference of Transformers upon MPC (ICML 2024 · similarity 0.87)
- CipherPrune: Efficient and Scalable Private Transformer Inference (ICLR 2025 · similarity 0.87)
- Exact Expressive Power of Transformers with Padding (NeurIPS 2025 · similarity 0.85)
- Transformers to SSMs: Distilling Quadratic Knowledge to Subquadratic Models (NeurIPS 2024 · similarity 0.84)
- The Expressive Power of Transformers with Chain of Thought (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.