Research map / Large language models / Language, safety and interpretability
Secure Outlier-Aware Large Language Model Inference
Lifan Zhao, Zhixuan Fang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CrypTen non-linear protocols with SOAL (prefixed special tokens, new RMSNorm/SiLU/Softmax protocols) | CrypTen · Llama2-7B, 512-token prompt | total time (s): 428.95 -> 193.59 | Improved |
| replace CrypTen non-linear protocols with SOAL | CrypTen · GPT-2, 512-token prompt | total time (s): 48.86 -> 23.30 | Improved |
| replace CrypTen non-linear protocols with SOAL | CrypTen · Mixtral 8x7B, 512-token prompt | total time (s): 1104.46 -> 668.23 | Improved |
| add conformant maxima to replace exact maximum finding in Softmax (ablation) | SOAL Softmax · Llama2-7B, 512-token prompt | SoftMax time (s): 199.99 -> 26.62 | Improved |
| add scaling secret and outlier removal to normalization (two Newton-Raphson iterations instead of 11) (ablation) | SOAL RMSNorm · Llama2-7B, 512-token prompt | Normalization time (s): 27.86 -> 14.87 | Improved |
| add special-token prefixing with new sigmoid protocol for activation (ablation) | SOAL SiLU · Llama2-7B, 512-token prompt | Activation time (s): 87.30 -> 37.13 | Improved |
| replace CrypTen normalization with SOAL on GPT-2 | SOAL RMSNorm · GPT-2, 512-token prompt | Normalization time (s): 3.07 -> 2.99 | No clear effect |
| replace CrypTen normalization with SOAL on Mixtral 8x7B communication | SOAL RMSNorm · Mixtral 8x7B, 512-token prompt | Normalization communication (GB): 24.31 -> 32.25 | Hurt |
| replace Sigma protocols with SOAL under function secret sharing | Sigma · GPT-2, 512 tokens | key size (GB): 86.686 -> 61.830 | Improved |
| replace Sigma protocols with SOAL under function secret sharing | Sigma · GPT-2, 512 tokens | online time (s): 10.048 -> 7.577 | Improved |
| add SOAL prefix tokens and protocols to Llama2-7B | Llama2-7B · WikiText-2 | perplexity: 5.55 -> 5.58 | No clear effect |
| add SOAL to GPT-2 | GPT-2 · WikiText-2 | perplexity: 36.25 -> 37.50 | Hurt |
Closest papers by meaning
- TensorCommitments: A Lightweight Verifiable Inference for Large Language Models (NeurIPS 2026 · similarity 0.87)
- TOPLOC: A Locality Sensitive Hashing Scheme for Trustless Verifiable Inference (ICML 2025 · similarity 0.86)
- Encryption-Friendly LLM Architecture (ICLR 2025 · similarity 0.86)
- Hidden No More: Attacking and Defending Private Third-Party LLM Inference (ICML 2025 · similarity 0.86)
- EncryptedLLM: Privacy-Preserving Large Language Model Inference via GPU-Accelerated Fully Homomorphic Encryption (ICML 2025 · similarity 0.86)
- Ditto: Quantization-aware Secure Inference of Transformers upon MPC (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
