Research map / Adversarial robustness and security / Attacks on text and graphs
SAME: Sample Reconstruction against Model Extraction Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAME, a model extraction attack detector that combines masked autoencoder reconstruction loss with an auxiliary model's prediction deviation, requiring no auxiliary OOD data, query history, or white-box access.
Problem
Detecting malicious query samples used in model extraction attacks against MLaaS models, under realistic defender constraints (no auxiliary OOD data, no user query history, no white-box access, no training intervention) and generalizing across unknown attack types.
Approach
Use a masked autoencoder's reconstruction loss to flag out-of-distribution queries, and add an auxiliary model trained on reconstructed data to mimic the victim; the disagreement between victim and auxiliary predictions catches perturbation-based attacks that reconstruction misses.
Key results
- On MNIST with a 1k budget, SAME reaches 93.30% AUROC against JBDA and 99.37% against Knockoff, well above OE and EDM.
- On CIFAR-10 with a 6k budget under Knockoff, SAME reaches 84.03% AUROC versus 77.38% for OE and 71.72% for EDM (the paper's text cites 92.53%, which does not match the table).
- SAME trains the victim to 83.28% accuracy in 170 s versus 80.83% (OE, 1563 s) and 80.37% (EDM, 2015 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SAME instead of the strongest listed baseline | Model extraction query detection on MNIST · MNIST, JBDA, 1k queries | AUROC (%): 66.06 -> 93.30 | Helped |
| Use SAME instead of the strongest listed baseline | Model extraction query detection on MNIST · MNIST, Knockoff, 1k queries | AUROC (%): 91.78 -> 99.37 | Helped |
| Use SAME instead of the strongest listed baseline | Model extraction query detection on MNIST · MNIST, DFME, 1k queries | AUROC (%): 100.00 -> 100.00 | No clear effect |
| Use SAME instead of the strongest listed baseline | Model extraction query detection on CIFAR-10 · CIFAR-10, JBDA, 6k queries | AUROC (%): 74.53 -> 80.89 | Helped |
| Use SAME instead of the strongest listed baseline | Model extraction query detection on CIFAR-10 · CIFAR-10, Knockoff, 6k queries | AUROC (%): 77.38 -> 84.03 | Helped |
| Use SAME instead of the strongest listed baseline | Model extraction query detection on CIFAR-10 · CIFAR-10, DFME, 6k queries | AUROC (%): 80.35 -> 97.64 | Helped |
| Train the victim with SAME rather than OE or EDM | Victim model · Victim model utility | Model accuracy (%): 80.83 -> 83.28; 80.37 -> 83.28 | Helped |
| Use SAME rather than OE or EDM | Victim model training · Victim model training | Training time (s): 1562.92 -> 170.44; 2014.80 -> 170.44 | Helped |
| Increase malicious-query prevalence | SAME · MNIST, malicious ratio 0.01 to 0.90 | AUPR (%): 82.78 -> 99.71 | Helped |
| Increase malicious-query prevalence | SAME · CIFAR-10, malicious ratio 0.01 to 0.90 | AUPR (%): 50.91 -> 95.98 | Helped |
| Remove the Auxiliary Model deviation-loss term, retaining only reconstruction loss (SAME-X) (ablation) | SAME · MNIST and CIFAR-10 detection | Detection performance | No clear effect |
| Remove the reconstruction-loss term, retaining only Auxiliary Model deviation loss (SAME-Y) (ablation) | SAME · MNIST and CIFAR-10 detection | Detection performance | No clear effect |
Surprising findings
- Reconstruction loss alone fails to separate perturbation-based (JBDA) queries from benign ones, motivating the auxiliary model.
- Classifier-based detectors with Outlier Exposure perform poorly against unseen attack distributions, with AUROC near 58-75% on some settings.
Limitations
- Evaluated only on small image datasets (MNIST and CIFAR-10) with a single victim architecture per dataset.
- Performance on CIFAR-10 is notably weaker (FPR95 around 8-57%) than on MNIST.
- Relies on hyperparameter alpha balancing the two score components and on MAE embedding size.
Experimental setup
Datasets: MNIST EMNIST-digits CIFAR-10 CIFAR-100 KMNIST Tiny ImageNet
Benchmarks: MNIST CIFAR-10
Metrics: AUROC AUPR FPR95
Baselines: Outlier Exposure (OE) Ensemble of Diverse Models (EDM) Prada
Builds on: Outlier Exposure Prada Masked Autoencoder KnockoffNets JBDA
Closest papers by meaning
- From Counterfactuals to Trees: Competitive Analysis of Model Extraction Attacks (NeurIPS 2025 · similarity 0.86)
- Data-Free Hard-Label Robustness Stealing Attack (AAAI 2024 · similarity 0.85)
- Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable (NeurIPS 2024 · similarity 0.85)
- A Sample-Level Evaluation and Generative Framework for Model Inversion Attacks (AAAI 2025 · similarity 0.85)
- HoneypotNet: Backdoor Attacks Against Model Extraction (AAAI 2025 · similarity 0.84)
- PubDef: Defending Against Transfer Attacks From Public Models (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.