Research map / Adversarial robustness and security / Black-box and transfer attacks
Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label Setting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QEDG, a data-free model stealing method under hard-label feedback that generates query-efficient synthetic samples near the substitute's decision boundary and adds query-free augmentation, evaluated on MNIST, FMNIST, SVHN, CIFAR10, CIFAR100 and Azure MLaaS.
Problem
Data-free model stealing against MLaaS classifiers that return only hard labels, where existing generators produce overconfident samples that the substitute already agrees with, so many queries are wasted and detection risk rises.
Approach
Generator loss combining a classification loss with a harmony loss (variance of the substitute's softmax outputs, pushing samples toward decision boundaries) and a diversity loss (cosine dissimilarity within a batch, spreading samples along the boundary); plus query-free augmentation that reuses target labels of non-disputed samples under simple flips/rotations, and a memory bank storing target la
Key results
- On MNIST, QEDG reaches 80.24% accuracy and 78.03% consistency with 5K queries versus 64.42%/65.61% for DFTA with 20K queries.
- On CIFAR10, QEDG reaches 43.86% accuracy with 30K queries versus 39.45% for DFTA with 400K... (table lists 400K for DFTA's CIFAR100 and 50K for CIFAR10).
- Ablation on CIFAR10 shows the full combination (QA + L_harm + L_div) gives the best accuracy 43.8%, consistency 39.4%, and ASR 72.9%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use QEDG instead of DFTA | model stealing · MNIST | accuracy (%): 64.42 -> 80.24 | Helped |
| Use QEDG instead of DFTA | model stealing · FMNIST | consistency (%): 41.48 -> 50.86 | Helped |
| Use QEDG instead of IDEAL | model stealing · SVHN | accuracy (%): 60.71 -> 66.58 | Helped |
| Use QEDG instead of DFTA | model stealing · CIFAR10 | consistency (%): 34.03 -> 39.45 | Helped |
| Use QEDG instead of DFME | model stealing · CIFAR100 | accuracy (%): 20.17 -> 22.32 | Helped |
| Add diversity loss to query-free augmentation plus harmony loss (ablation) | QEDG · CIFAR10 | accuracy (%): 31.2 -> 43.8 | Helped |
| Add diversity loss to query-free augmentation plus harmony loss (ablation) | QEDG · CIFAR10 | consistency (%): 25.7 -> 39.4 | Helped |
| Add query-free augmentation to harmony loss plus diversity loss (ablation) | QEDG · CIFAR10 | accuracy (%): 38.8 -> 43.8 | Helped |
| Add query-free augmentation to harmony loss plus diversity loss (ablation) | QEDG · CIFAR10 | consistency (%): 33.4 -> 39.4 | Helped |
| Add query-free augmentation and diversity loss instead of harmony loss alone (ablation) | QEDG · CIFAR10 | accuracy (%): 23.1 -> 41.9 | Helped |
| Add query-free augmentation and diversity loss instead of harmony loss alone (ablation) | QEDG · CIFAR10 | consistency (%): 14.8 -> 37.3 | Helped |
| Add diversity loss to query-free augmentation alone (ablation) | QEDG · CIFAR10 | accuracy (%): 31.9 -> 41.9 | Helped |
Surprising findings
- Using L_harm alone without L_div performs worst in the ablation, which the authors attribute to negative optimization.
Limitations
- Generated samples lack semantic information, as the authors acknowledge.
- Hyperparameters alpha, beta, gamma must be balanced across loss scales; performance degrades if loss magnitudes differ or gamma is too high.
Experimental setup
Datasets: MNIST FMNIST SVHN CIFAR10 CIFAR100
Benchmarks: MNIST FMNIST SVHN CIFAR10 CIFAR100 Microsoft Azure Custom Vision MNIST
Metrics: accuracy consistency rate Cohen's kappa attack success rate (FGSM, BIM, PGD)
Baselines: HEE DaST DFME IDEAL DFTA
Builds on: DFTA IDEAL DaST DFME HEE
Closest papers by meaning
- Stealix: Model Stealing via Prompt Evolution (ICML 2025 · similarity 0.86)
- From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks (NeurIPS 2025 · similarity 0.85)
- Sim4Rec: Data-Free Model Extraction Attack on Sequential Recommendation (AAAI 2025 · similarity 0.84)
- Data-Free Hard-Label Robustness Stealing Attack (AAAI 2024 · similarity 0.84)
- Adversarial Machine Unlearning (ICLR 2025 · similarity 0.84)
- Defense Against Model Stealing Based on Account-Aware Distribution Discrepancy (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.