Research map / Large language models / Language, safety and interpretability
DOMBA: Double Model Balancing for Access-Controlled Language Models via Minimum-Bounded Aggregation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DOMBA, which trains two submodels on partitions of access levels and aggregates their relative probabilities with a min-bounded function to limit exposure of restricted information while keeping utility.
Problem
Training LLMs on access-controlled organizational data risks exposing restricted information to unauthorized users; per-access-level models lose utility due to limited data, and existing exposure-limiting methods fail when sensitive info recurs across many examples.
Approach
Train two submodels on disjoint partitions of access levels, then at inference aggregate their relative token probabilities with a min-bounded mean (e.g., harmonic mean or minimum), so tokens known to only one submodel are suppressed; a token-exposure framework bounds the resulting leakage.
Key results
- DOMBA achieves maximum exposure of 4.69, versus 8.5e4 for SUBMIX and 1.3e10 for AGG-A.
- DOMBA obtains the best AUPPL on the recipe dataset (16.85 vs 27.87 for Per-AL) and SIA of 0.54 (recipes) / 0.62 (movies) among secure models.
- Non-secure FT-ALL achieves the best utility (HOPPL 19.9 recipes) but much worse security (SIA 0.81, CAN 14.47 on recipes).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the second submodel on each access level and use minimum aggregation, rather than training separate per-access-level models | DOMBA vs Per-AL · Recipes and Movie reviews | HOPPL: 45.15 -> 25.19; 63.39 -> 52.22 | Helped |
| Fine-tune the second submodel on each access level and use minimum aggregation, rather than training separate per-access-level models | DOMBA vs Per-AL · Recipes and Movie reviews | AUPPL: 27.87 -> 16.85; 54.18 -> 42.48 | Helped |
| Add DOMBA-FT to DOMBA-INIT (ablation) | DOMBA · Recipes and Movie reviews | AUPPL: 21.14 -> 16.85; 49.79 -> 42.48 | Helped |
| Use minimum rather than harmonic-mean aggregation (ablation) | DOMBA-INIT · Recipes and Movie reviews | EXP: 2.38 -> 1.77; 2.84 -> 2.31 | Helped |
| Use minimum rather than harmonic-mean aggregation (ablation) | DOMBA-INIT · Recipes and Movie reviews | HOPPL: 23.57 -> 24.54; 50.83 -> 51.89 | Hurt |
| Replace minimum aggregation with harmonic mean (ablation) | DOMBA · Recipes and Movie reviews | Security and utility metrics | Mixed |
| Increase training epochs (ablation) | DOMBA, D-I-H, and D-I-M · Recipes and Movie reviews | Exposure | Hurt |
| Increase training epochs (ablation) | DOMBA and other methods · Recipes and Movie reviews | Utility | Helped |
Surprising findings
- Harmonic-mean and minimum aggregation perform similarly for DOMBA, and the minimum version gives better security but slightly worse utility than harmonic mean in the DOMBA-INIT ablation.
Limitations
- Requires access-level partitions with no shared sensitive information (disconnected secret-sharing graph), which may not hold in practice.
- Inference cost of running two LLMs; distillation suggested as remedy.
- Experiments use a small 117M-parameter GPT and only two datasets with proxy sensitive phrases.
Experimental setup
Datasets: IMDB Spoiler reviews Movies Metadata dataset Food.com Recipes and Interactions dataset Food-101 class labels
Benchmarks: IMDB Spoiler reviews (movie reviews split) Food.com recipes (recipe split)
Metrics: HOPPL perplexity AUPPL perplexity exposure (max and 99th percentile) secret perplexity secret inference attack AUC-ROC canary extraction score
Baselines: FT-ALL AGG-A SUBMIX Per-AL
Builds on: SUBMIX LoRA membership inference attack with reference model canary technique
Closest papers by meaning
- Scaling Laws for Differentially Private Language Models (ICML 2025 · similarity 0.87)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.87)
- Open LLMs are Necessary for Current Private Adaptations and Outperform their Closed Alternatives (NeurIPS 2024 · similarity 0.86)
- SILO Language Models: Isolating Legal Risk In a Nonparametric Datastore (ICLR 2024 · similarity 0.86)
- Permissioned LLMs: Enforcing Access Control in Large Language Models (NeurIPS 2025 · similarity 0.86)
- Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.