Research map / Adversarial robustness and security / Attacks on text and graphs
Confidence Is All You Need for MI Attacks (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a membership inference attack that replaces LiRA's loss-based signal with softmax confidence (true-class or argmax, label-free variant) fit with Gaussian IN/OUT distributions and a likelihood ratio test.
Problem
Membership inference attacks typically rely on loss values and the ground-truth label of the target point, which limits their effectiveness and practicality.
Approach
Uses log softmax confidence (for the true class, or the argmax class for a label-free variant) instead of logit-scaled loss in a LiRA-style likelihood ratio test over shadow IN/OUT models.
Key results
- Online confidence-based attack with fixed variance reaches AUC of about 0.601 (log confidence) versus 0.575 for the loss-value baseline on CIFAR-10.
- Label-free argmax variant reaches AUC of about 0.562 (online, fixed variance), comparable to the label-based variants.
- Offline attack AUCs are around 0.47 to 0.52, below the loss baseline of 0.518.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace loss values with confidence values | Membership inference attack · Attack (Online) | AUC: 0.5753 -> 0.5668 | Hurt |
| Replace loss values with log confidence values | Membership inference attack · Attack (Online) | AUC: 0.5753 -> 0.575 | No clear effect |
| Replace loss values with confidence values | Membership inference attack with fixed variance · Attack (Online,FV) | AUC: 0.5879 -> 0.593 | Helped |
| Replace loss values with log confidence values | Membership inference attack with fixed variance · Attack (Online,FV) | AUC: 0.5879 -> 0.6009 | Helped |
| Replace loss values with confidence values | Membership inference attack · Attack (Offline) | AUC: 0.5181 -> 0.492 | Hurt |
| Replace loss values with log confidence values | Membership inference attack · Attack (Offline) | AUC: 0.5181 -> 0.4721 | Hurt |
| Replace loss values with confidence values | Membership inference attack with fixed variance · Attack (Offline,FV) | AUC: 0.5184 -> 0.4928 | Hurt |
| Replace loss values with log confidence values | Membership inference attack with fixed variance · Attack (Offline,FV) | AUC: 0.5184 -> 0.4804 | Hurt |
| Replace loss values with log confidence values | Global-threshold membership inference attack · Attack Global Threshold | AUC: 0.5448 -> 0.5469 | No clear effect |
| Use argmax confidence instead of ground-truth-class confidence | Membership inference attack · Attack (Online) | AUC: 0.5753 -> 0.5464 | Hurt |
| Use argmax confidence instead of ground-truth-class confidence | Membership inference attack · Attack (Offline) | AUC: 0.5181 -> 0.478 | Hurt |
| Use argmax confidence instead of ground-truth-class confidence | Global-threshold membership inference attack · Attack Global Threshold | AUC: 0.5448 -> 0.5376 | Hurt |
Surprising findings
- Offline confidence-based attacks perform slightly below or near the loss baseline, with several values under 0.5.
Limitations
- Limited compute restricted training to 21 epochs and a small number of shadow models (16).
- Offline results are not clearly better than the loss baseline.
- Results are reported only with AUC on a single dataset in a short abstract.
Experimental setup
Datasets: CIFAR-10
Benchmarks: CIFAR-10
Metrics: AUC
Baselines: LOSS attack LiRA (loss-based likelihood ratio test) Attack Global Threshold
Builds on: LiRA LOSS attack Shokri et al. membership inference
Closest papers by meaning
- Low-Cost High-Power Membership Inference Attacks (ICML 2024 · similarity 0.88)
- Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration (NeurIPS 2024 · similarity 0.86)
- Estimating Model-Level Membership Inference Vulnerability Without Reference Models (NeurIPS 2026 · similarity 0.86)
- Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning (ICLR 2024 · similarity 0.86)
- Impact of Dataset Properties on Membership Inference Vulnerability of Deep Transfer Learning (NeurIPS 2025 · similarity 0.85)
- Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.