Research map / Large language models / Instruction and fine-tuning
Exploring the limits of strong membership inference attacks on large language models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scales LiRA to GPT-2 models (10M to 1B parameters) trained on up to ~100M C4 samples with thousands of reference models, finding strong MIAs succeed but remain limited (AUC<0.7 in typical settings), with substantial per-sample decision instability.
Problem
Whether membership inference attacks are fundamentally ineffective on pre-trained LLMs, given that strong attacks need many reference models that are costly to train at LLM scale.
Approach
Pre-trains thousands of GPT-2-style reference models on very large C4 subsets so the strongest reference-based attack (LiRA) can be run at pre-trained LLM scale, and measures per-sample decision stability via a flip-rate diagnostic across training seeds.
Key results
- LiRA on a 140M model trained on ~7M samples reaches AUC 0.678 with 128 references and never reaches 0.7 across reference counts.
- Under Chinchilla-optimal 1-epoch training, AUC ranges from 0.547 (489M) to 0.699 (85M); AUC rises to 0.797 for 140M after 10 epochs.
- For a 302M model, roughly 15.4% of true positives at FPR=1e-3 behave like coin flips (flip rate >= 0.487), rising to 42.2% when including flip rate >= 0.4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase the number of IN references from 4 to 64 (ablation) | LiRA on GPT-2 140M · C4, approximately 7M samples | AUC: 0.594 -> 0.678 | Helped |
| use loss instead of logits as the observation statistic (ablation) | LiRA on GPT-2 140M · C4, approximately 7M samples | AUC: 0.576 -> 0.678 | Helped |
| train for 2 epochs on half the dataset rather than 1 epoch on the full dataset (ablation) | LiRA on GPT-2 44M · C4, Chinchilla-optimal total training tokens | AUC: 0.620 -> 0.744 | Helped |
| continue training from epoch 1 through epoch 10 (ablation) | LiRA on GPT-2 140M · C4, Chinchilla-optimal dataset | AUC: 0.573 -> 0.797 | Helped |
| train on around 1M samples rather than very small or very large datasets (ablation) | LiRA on GPT-2 140M · C4, 1 epoch | AUC: 0.753 | Mixed |
| increase model size while keeping the training set fixed at approximately 8.3M samples (ablation) | LiRA · C4, approximately 8.3M samples, 1 epoch | TPR at fixed FPR | Helped |
| increase model size under Chinchilla-optimal 1-epoch training (ablation) | LiRA · C4, Chinchilla-optimal training | AUC: 0.699; 0.689; 0.547; 0.654; 0.553 | Mixed |
| use LiRA rather than RMIA with 64 IN references | membership inference on GPT-2 140M · C4, approximately 7M samples | AUC: 0.658 -> 0.678 | Helped |
| use RMIA rather than LiRA with 4 IN references | membership inference on GPT-2 140M · C4, approximately 7M samples | AUC: 0.594 -> 0.643 | Helped |
| increase training dataset size from the Chinchilla-optimal setting to approximately 8.3M samples while holding model size fixed (ablation) | LiRA on GPT-2 models · C4, 1 epoch | TPR at fixed FPR | Mixed |
| remove the population comparison from RMIA and use its simple variant (ablation) | RMIA on GPT-2 10M · C4, 2^19 training samples, 1 epoch | AUC | No clear effect |
| increase RMIA population size |Z| (ablation) | RMIA on GPT-2 10M · C4, 2^19 training samples, 1 epoch | TPR at low FPR | Helped |
Surprising findings
- Larger models are not necessarily more vulnerable: under Chinchilla-optimal training, the 489M model has lower AUC (0.547) than the 85M and 302M models.
- Discoverable/probabilistic extraction shows no correlation with MIA success; the most MIA-vulnerable samples are rarely extractable.
Limitations
- Results are for GPT-2-style models trained on C4 and may not transfer to other architectures or data.
- Flip-rate diagnostic uses varied targets rather than the single-target threat model an attacker faces.
- Chinchilla scaling is only one token-selection rule; other scaling laws were not tested.
Experimental setup
Datasets: C4
Benchmarks: C4 member/non-member splits
Metrics: AUC TPR at fixed FPR flip rate
Baselines: loss attack RMIA fine-tuning attacks
Builds on: LiRA RMIA NanoDO Chinchilla scaling laws
Closest papers by meaning
- Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration (NeurIPS 2024 · similarity 0.93)
- MIA-Tuner: Adapting Large Language Models as Pre-training Text Detector (AAAI 2025 · similarity 0.92)
- DF-MIA: A Distribution-Free Membership Inference Attack on Fine-Tuned Large Language Models (AAAI 2025 · similarity 0.91)
- Membership Inference Attacks against Large Vision-Language Models (NeurIPS 2024 · similarity 0.89)
- Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.88)
- Estimating Model-Level Membership Inference Vulnerability Without Reference Models (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.