Research map / Large language models / Instruction and fine-tuning
Permissioned LLMs: Enforcing Access Control in Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Permissioned LLMs (PermLLM) that enforce domain-level access control by mapping security domains to LoRA adapters (Activate, Merge, Union), and audits them with access-advantage metrics DDI (membership inference) and UGI (utility gap).
Problem
LLMs fine-tuned on siloed enterprise data can leak information across access-control domains to users lacking privileges; existing prompt-based or encrypted-tag defenses are not absolute and lack formal auditing.
Approach
Superimpose organizational access control on LLM responses by routing each security domain's training data to dedicated PEFT (LoRA) adapters and activating only the adapters a user is authorized for; formalize "relevant response" and "access advantage", and instantiate access advantage as DDI (MIA-based) and UGI (utility-gap-based) audit metrics.
Key results
- Activate achieves near-perfect DDI (AUC-ROC around 0.99-1.00 for single domain with Loss/Ref MIAs) on most datasets, indicating an auditor can almost perfectly identify the active domain.
- Utility Gap Index for single domain is far below 1 (e.g., Delta_acc around 0.2-0.5 on WMDP/SimpleQA; near zero on PubMedQA), and Union keeps the largest gaps when users access two domains (e.g., Delta_acc 0.59 on SimpleQA for Llama).
- Union retains the highest utility gaps and DDI when combining domains, while Activate and Merge lose utility as the number of combined domains grows.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train a separate adapter on the union of the authorized domains instead of activating separate adapters (ablation) | PermLLM · GPQA, SimpleQA, and WMDP (two domains) | Utility Gap Index: GPQA Δbleu: 0.15 ± 0.06 -> 0.51 ± 0.07; SimpleQA Δbleu: 0.26 ± 0.09 -> 0.61 ± 0.03; WMDP Δacc: 0.27 ± 0.07 -> 0.34 ± 0.1 | Helped |
| Merge the authorized-domain adapters instead of activating them separately (ablation) | PermLLM · WMDP, GPQA, and SimpleQA (two domains) | Utility Gap Index: WMDP Δacc: 0.27 ± 0.07 -> 0.21 ± 0.09; GPQA Δbleu: 0.15 ± 0.06 -> 0.11 ± 0.06; SimpleQA Δbleu: 0.26 ± 0.09 -> 0.23 ± 0.0 | Hurt |
| Use Union rather than Merge for two-domain access (ablation) | PermLLM · GPQA, SimpleQA, and WMDP (two domains) | Utility Gap Index: GPQA Δbleu: 0.11 ± 0.06 -> 0.51 ± 0.07; SimpleQA Δbleu: 0.23 ± 0.09 -> 0.61 ± 0.03; WMDP Δacc: 0.21 ± 0.09 -> 0.34 ± 0.1 | Helped |
| Combine domain adapters for access to multiple domains instead of serving a single domain (ablation) | Activate · Multiple-domain access | model utility | Hurt |
| Use domains derived by k-means clustering rather than distinct source distributions | PermLLM · PubMedQA | Utility Gap Index: Δbleurt: 0.01 ± 0.00 (Activate), 0.00 ± 0.00 (Merge), 0.01 ± 0.00 (Union) | Hurt |
Limitations
- Does not support deep hierarchies of domains with arbitrary overlaps.
- Scalability beyond a handful of domains: Activate degrades utility and Union becomes compute-intensive.
- DDI is only as good as the strongest membership inference attack used.
Experimental setup
Datasets: WMDP GPQA SimpleQA RCV1 PubMedQA
Benchmarks: WMDP GPQA SimpleQA RCV1 PubMedQA
Metrics: AUC-ROC TPR@1%FPR TPR@5%FPR BLEURT BERTScore F1 SacreBLEU verbatim accuracy Utility Gap Index Domain Distinguishability Index
Baselines: full fine-tuning LoRA fine-tuning prompt-based access control
Builds on: LoRA TIES DARE membership inference attacks
Closest papers by meaning
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.88)
- PrivAuditor: Benchmarking Data Protection Vulnerabilities in LLM Adaptation Techniques (NeurIPS 2024 · similarity 0.87)
- AC-LoRA: (Almost) Training-Free Access Control Aware Multi-Modal LLMs (NeurIPS 2025 · similarity 0.86)
- DOMBA: Double Model Balancing for Access-Controlled Language Models via Minimum-Bounded Aggregation (AAAI 2025 · similarity 0.86)
- Can LLMs Keep a Secret? Testing Privacy Implications of Language Models via Contextual Integrity Theory (ICLR 2024 · similarity 0.86)
- No More, No Less: Least-Privilege Language Models (ICML 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.