Research map / Adversarial robustness and security / Attacks on text and graphs
Chameleon: Increasing Label-Only Membership Leakage with Adaptive Poisoning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Chameleon, a label-only membership inference attack that uses adaptive data poisoning (poison replicas added per challenge point until shadow OUT models misclassify it) and a 64-query neighborhood-based misclassification score, greatly improving low-FPR TPR over prior label-only attacks.
Problem
Label-only membership inference is ineffective at low false positive rates, and naive poisoning (static label flipping from Truth Serum) fails to help in this setting.
Approach
Adaptively choose the number of poisoned label-flipped replicas per challenge point so that OUT shadow models misclassify it, then score membership via the target model's label predictions on a small set of KL-close neighbors (64 queries), enabling high TPR at low FPR with few queries.
Key results
- On CIFAR-10, Chameleon achieves 22.8% TPR@1%FPR versus 1.3% for the Decision-Boundary attack, a roughly 17.5x improvement.
- Chameleon uses 64 target queries per challenge point versus about 2,500 for the Decision-Boundary attack (about 39x more query-efficient).
- Under DP-SGD with epsilon=4, Chameleon's TPR@1%FPR drops to 0% while CIFAR-10 accuracy falls from 84.3% to 49.4%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Chameleon instead of Decision-Boundary | label-only membership inference · GTSRB | TPR@1% FPR: 1.1% -> 11.4% | Helped |
| use Chameleon instead of Decision-Boundary | label-only membership inference · CIFAR-10 | TPR@1% FPR: 1.3% -> 22.8% | Helped |
| use Chameleon instead of Decision-Boundary | label-only membership inference · CIFAR-100 | TPR@1% FPR: 3.6% -> 52.5% | Helped |
| replace adaptive poisoning with static poisoning (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR: 22.9% -> 8.1% | Hurt |
| add poisoning rather than use no poisoning (ablation) | Chameleon · CIFAR-10 | AUC | Helped |
| set the poisoning threshold close to 0, causing overpoisoning (ablation) | Chameleon · CIFAR-10 | TPR | Hurt |
| increase the number of OUT models from 1 to 8 (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR: 13.1% -> 22.9% | Helped |
| increase the number of OUT models from 8 to 64 (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR: 22.9% -> 24.1% | Helped |
| increase maximum poisoned iterations (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR | Helped |
| increase membership neighborhood size from 16 to 64 (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR: +0.6% | Helped |
| increase neighborhood threshold from 0.25 to 1.75 (ablation) | Chameleon · CIFAR-10 | TPR@1% FPR: 23.5% -> 17.3% | Hurt |
| train the target model with DP-SGD at ε=100 instead of ε=∞ | Chameleon · CIFAR-10 | TPR@1% FPR: 22.6% -> 6.1% | Hurt |
Surprising findings
- Naively adding Truth Serum's static label-flipping poisoning to the Decision-Boundary label-only attack reduces its TPR at 1% FPR to 0%.
Limitations
- Requires training many shadow models (e.g., 16 to 80 ResNet-18 models) and adds poisoned points to the training data, which imposes burden on the adversary.
- Attack effectiveness drops to zero under DP-SGD with strict privacy budgets (epsilon=4), though with large utility loss.
- Still relies on poisoning; the authors note that effective low-FPR label-only attacks without poisoning remain open.
Experimental setup
Datasets: GTSRB CIFAR-10 CIFAR-100 Purchase-100
Benchmarks: GTSRB CIFAR-10 CIFAR-100 Purchase-100
Metrics: TPR@0.1%FPR TPR@1%FPR TPR@5%FPR TPR@10%FPR AUC MI accuracy test accuracy
Baselines: Gap attack Decision-Boundary attack Truth Serum static poisoning LiRA
Builds on: Truth Serum LiRA Decision-Boundary label-only MI attack DP-SGD
Closest papers by meaning
- Low-Cost High-Power Membership Inference Attacks (ICML 2024 · similarity 0.89)
- OSLO: One-Shot Label-Only Membership Inference Attacks (NeurIPS 2024 · similarity 0.87)
- FedMIA: An Effective Membership Inference Attack Exploiting "All for One" Principle in Federated Learning (CVPR 2025 · similarity 0.87)
- What Really is a Member? Discrediting Membership Inference via Poisoning (NeurIPS 2025 · similarity 0.87)
- Assessing Per-Sample Membership Inference Vulnerability without Retraining (NeurIPS 2026 · similarity 0.87)
- Causal Evaluation of Membership Inference Attacks (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.