Research map / Architectures and efficiency / RNNs and normalization
MOAI: Module-Optimizing Architecture for Non-Interactive Secure Transformer Inference
Linru Zhang, Xiangning Wang, Jun Jie Sim, Zhicong Huang, Jiahao Zhong, HUAXIONG WANG, Pu Duan, Kwok-Yan Lam
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use consistent column/diagonal packing with rotation-free Softmax/LayerNorm and interleaved batching (MOAI) instead of THOR | THOR · BERT-base on A100 | total time (s): 602.26 -> 283.95 (52.8% reduction) | Improved |
| apply MOAI evaluation flow to Powerformer's FHE-friendly model | Powerformer · BERT-base on A100 | total time (s): 344.52 -> 152.4 (55.7% reduction) | Improved |
| use rotation-free Softmax and LayerNorm (ablation) | MOAI vs NEXUS/THOR · BERT-base | HE rotations: 2448 -> 0 | Improved |
| add column packing and interleaved batching for matrix multiplication | THOR · BERT-base | HE rotations in matmuls: 22224 -> 9648 | Improved |
| MOAI vs Powerformer matmul rotations | Powerformer · BERT-base | HE rotations in matmuls: 16740 -> 9648 | Improved |
| run MOAI under FHE encryption instead of plaintext | BERT-base · SST-2 | accuracy: 93.9 -> 93.8 | No clear effect |
| run MOAI under FHE encryption instead of plaintext | BERT-base · QNLI | accuracy: 90.7 -> 90.5 | No clear effect |
| run MOAI under FHE encryption instead of plaintext | BERT-base · RTE | accuracy: 65.7 -> 65.5 | No clear effect |
| bootstrap ctsum after the sum step in Softmax to lower the Softmax depth (ablation) | MOAI Softmax · BERT-base | Softmax depth: 20 -> 10 layers | Improved |
| keep Powerformer pooler/classification (not optimized by MOAI) | MOAI on Powerformer · BERT-base on A100 | Pooler & Classification time (s): 0.2 -> 0.7 | Hurt |
Closest papers by meaning
- Converting Transformers to Polynomial Form for Secure Inference Over Homomorphic Encryption (ICML 2024 · similarity 0.92)
- Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.91)
- Nimbus: Secure and Efficient Two-Party Inference for Transformers (NeurIPS 2024 · similarity 0.89)
- CipherPrune: Efficient and Scalable Private Transformer Inference (ICLR 2025 · similarity 0.89)
- EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption (ICML 2025 · similarity 0.87)
- Ditto: Quantization-aware Secure Inference of Transformers upon MPC (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
