Research map / Adversarial robustness and security / Attacks on text and graphs
Low-Cost High-Power Membership Inference Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RMIA, a low-cost membership inference attack built on a pairwise Bayesian likelihood-ratio test against population samples, which works offline with few reference models and achieves higher TPR at low FPR than LiRA, Attack-R, Attack-P and Quantile Regression.
Problem
Membership inference attacks are expensive (many reference models, often per-query online training) and unstable across settings such as few reference models, distribution shift, architecture mismatch, and OOD non-members, and they perform poorly at low FPR.
Approach
Models the null hypothesis as the target point being replaced by random population samples, composes pairwise likelihood-ratio tests LR(x,z)=Pr(x|θ)/Pr(x) ÷ Pr(z|θ)/Pr(z) computed via Bayes rule, and scores x as the fraction of population z it γ-dominates; the offline variant approximates the IN-model normalizer from OUT models with a linear scaling factor a.
Key results
- With only 1 reference model, RMIA shows at least 24% higher AUC than LiRA across datasets, and about 2x-4x higher TPR at low FPR using 1-2 reference models.
- With 4 CIFAR-10 reference models, RMIA has at least 6% higher AUC and about 3x higher TPR at zero FPR than LiRA (Online).
- Using 254 models with 18 augmented queries, RMIA achieves about 4x TPR at zero FPR and about 4.6% higher AUC than LiRA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RMIA instead of prior attacks with few reference models | Membership inference attacks · CIFAR-10, 2 reference models | AUC: around 10% higher AUC than Attack-R and LiRA | Helped |
| Use RMIA offline instead of LiRA online | Membership inference attacks · CIFAR-10, 4 reference models | AUC: at least 6% higher AUC | Helped |
| Use RMIA with one reference model instead of LiRA | Membership inference attacks · All datasets, 1 reference model | AUC: at least 24% higher AUC | Helped |
| Use RMIA offline instead of Attack-R | Membership inference attacks · All datasets, 127 OUT reference models | AUC: 5%-10% lower AUC than RMIA | Helped |
| Use RMIA rather than Attack-R for OOD non-members | Offline membership inference · CIFAR-10 target, CINIC-10 OOD non-members | AUC: at least 21% higher AUC | Helped |
| Increase the number of population comparison samples z (ablation) | RMIA (online) · CIFAR-10 | AUC: 2500 population samples | Helped |
| Use 2500 population samples instead of a 10-fold larger population set (ablation) | RMIA (online) · CIFAR-10 | AUC: 2500 population samples; 10 fold larger population set | No clear effect |
| Use multiple augmented queries with majority voting (ablation) | RMIA · 18 augmented queries | AUC: about 4.6% higher AUC compared to LiRA | Helped |
| Use multiple augmented queries with majority voting (ablation) | RMIA · 18 augmented queries | TPR at 0% FPR: 4× improvement in TPR at zero FPR | Helped |
| Use SM-Taylor-Softmax instead of other confidence functions (ablation) | RMIA on CIFAR-10 · CIFAR-10 | AUC | Helped |
| Vary Taylor order, soft-margin, or temperature in SM-Taylor-Softmax (ablation) | RMIA on CIFAR-10 · CIFAR-10 | AUC | No clear effect |
| Set the offline approximation parameter a closer to 1 (ablation) | RMIA (offline) · CIFAR-10 | AUC: up to 5.5% | Hurt |
Surprising findings
- Offline RMIA with 127 OUT models performs comparably to online attacks, and offline RMIA beats LiRA (Online) unless hundreds of reference models are used.
- Offline LiRA reproduced with the authors' code achieves much lower power than reported in prior work.
Limitations
- Online variants require training reference models per query, which is impractical; the authors treat them as proof-of-concept.
- Offline Pr(x) normalizer relies on a heuristic scaling factor a tuned per dataset (e.g., 0.3 for CIFAR-10, 1 for ImageNet).
- Performance gains are reported mainly for image and tabular classification with a specific confidence function (SM-Taylor-Softmax).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 CINIC-10 ImageNet Purchase-100
Benchmarks: CIFAR-10 CIFAR-100 CINIC-10 ImageNet Purchase-100
Metrics: AUC TPR at 0.01% FPR TPR at 0% FPR ROC curve
Baselines: LiRA LiRA (Online) Attack-R Attack-P Quantile Regression LOSS attack
Builds on: LiRA Attack-R Attack-P Quantile Regression MIA
Closest papers by meaning
- Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning (ICLR 2024 · similarity 0.89)
- Estimating Model-Level Membership Inference Vulnerability Without Reference Models (NeurIPS 2026 · similarity 0.89)
- Confidence Is All You Need for MI Attacks (Student Abstract) (AAAI 2024 · similarity 0.88)
- Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.88)
- Exploring the limits of strong membership inference attacks on large language models (NeurIPS 2025 · similarity 0.87)
- Assessing Per-Sample Membership Inference Vulnerability without Retraining (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.