Research map / Adversarial robustness and security / Attacks on text and graphs

What Really is a Member? Discrediting Membership Inference via Poisoning

NeurIPS 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Membership inference and inversion · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that membership inference tests remain unreliable under neighborhood-based membership definitions: dataset poisoning (PoisonM) substitutes training points to flip MI test predictions, driving AUC well below random on Pythia fine-tuning.

Problem

Membership inference tests for LLMs are unreliable; even under relaxed neighborhood-based membership definitions, an adversary who poisons the training data can make the test mispredict membership for a target point.

Approach

Formalizes a robustness notion for MI tests under membership-invariant dataset substitutions, proves a trade-off between a test's advantage on clean data and its robustness to poisoning, and gives PoisonM, a model-agnostic greedy token-substitution attack that creates poisoned neighbors and non-neighbors of a target point.

Key results

• PoisonM reduces LOSS-test AUC on AI4Privacy with 7-gram neighborhood from 0.587 to 0.252 and reference-based test from 0.647 to 0.089.
• Under exact match, reference-based test AUC on AI4Privacy drops from 0.625 to 0.022 under PoisonM.
• Dataset inference fails under poisoning, yielding incorrect p-value predictions for members and non-members in the 7-gram and embedding settings.

Measured effects reported in the paper

Surprising findings

• The tests that perform best on clean data are the most vulnerable under poisoning, consistent with the theoretical trade-off.
• Simple baselines (token dropouts, casing flips, chunking) can also push AUC below random in some settings, though PoisonM is consistently stronger.

Limitations

• Poisoned non-neighbor generation requires multiple poisons (b2=10), a limitation the authors acknowledge.
• Experiments focus on fine-tuning with Pythia models and two canary datasets; pre-training not evaluated due to compute cost.
• Evaluation limited to specific neighborhood parameters and a fixed set of MI tests.

Experimental setup

Datasets: Wikitext-103 AI4Privacy AGNews

Benchmarks: AI4Privacy canary set AGNews canary set

Metrics: AUC TPR@1% FPR dataset inference p-values

Baselines: Token Dropouts Casing Flips Chunk insertion (adapted from Liu et al.) Perplexity/LOSS kmin zlib perturbation test reference-based test dataset inference

Builds on: Zou et al. greedy coordinate descent attack Min-K% Prob dataset inference Maini et al. membership inference evaluation

Closest papers by meaning

• Membership Inference Attacks against Fine-tuned Large Language Models via Self-prompt Calibration (NeurIPS 2024 · similarity 0.88)
• LLM Dataset Inference: Did you train on my dataset? (NeurIPS 2024 · similarity 0.88)
• Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning (ICLR 2024 · similarity 0.87)
• Exploring the limits of strong membership inference attacks on large language models (NeurIPS 2025 · similarity 0.87)
• DF-MIA: A Distribution-Free Membership Inference Attack on Fine-Tuned Large Language Models (AAAI 2025 · similarity 0.86)
• Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.86)