Research map / Image generation / Autoregressive and latent generation
Stealix: Model Stealing via Prompt Evolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Stealix steals image classifiers via black-box queries without human-crafted prompts, evolving prompts with a PEZ-style contrastive refinement plus a genetic algorithm guided by victim hard-label feedback (prompt consistency).
Problem
Model stealing of image classifiers using pre-trained diffusion models currently relies on class names or manually crafted prompts, which is limiting for attackers lacking prompt expertise or domain knowledge; the paper studies a prompt-free, black-box threat model with one seed image per class and
Approach
Treat prompt design as an evolutionary search: image triplets (seed, positive, negative) drive contrastive prompt optimization via CLIP-style encoders, and prompts are scored by prompt consistency (fraction of synthetic images the victim labels as the target class), then evolved by tournament selection, crossover and mutation.
Key results
- With 500 queries per class, Stealix reaches 65.9% on EuroSAT, 40.0% on PASCAL, 49.6% on CIFAR10 and 48.4% on DomainNet, outperforming DA-Fusion and Real Guidance across all four datasets.
- On CIFAR10 Stealix attains 49.6% vs 27.4% for Real Guidance (22.2-point gain), and on the NSFW victim it reaches 73.0% vs 50.0% for Real Guidance.
- Prompt consistency correlates negatively with L2 feature distance to victim data (Spearman rho from -0.63 to -0.88 across datasets, all p<0.001).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace DA-Fusion with Stealix prompt evolution | model stealing with Stable Diffusion-v2 and 500 queries per class · EuroSAT | attacker accuracy: 59.0% -> 65.9% | Helped |
| replace ASPKD with Stealix prompt evolution | model stealing with 500 queries per class · PASCAL | attacker accuracy: 25.7% -> 40.0% | Helped |
| replace Real Guidance with Stealix prompt evolution | model stealing with 500 queries per class · CIFAR10 | attacker accuracy: 27.4% -> 49.6% | Helped |
| replace Knockoff Nets with Stealix prompt evolution | model stealing with 500 queries per class · DomainNet | attacker accuracy: 39.3% -> 48.4% | Helped |
| add prompt reproduction to seed-only prompt optimization (ablation) | Stealix · EuroSAT | attacker accuracy: 60.1% -> 65.9% | Helped |
| add prompt reproduction to seed-only prompt optimization (ablation) | Stealix · PASCAL | attacker accuracy: 26.7% -> 40.0% | Helped |
| add prompt reproduction to seed-only prompt optimization (ablation) | Stealix · CIFAR10 | attacker accuracy: 33.8% -> 49.6% | Helped |
| add prompt reproduction to seed-only prompt optimization (ablation) | Stealix · DomainNet | attacker accuracy: 39.2% -> 48.4% | Helped |
| set prompt length to 16 rather than 4 or 32 (ablation) | Stealix · EuroSAT | attacker accuracy: 62.5%, 65.9%, and 64.3% | Mixed |
| add victim-model queries and labels to DA-Fusion training | DA-Fusion · EuroSAT | attacker accuracy: 29.9% -> 59.0% | Helped |
| replace human-readable InstructBLIP prompts with Stealix prompts | prompt-based stealing on EuroSAT · EuroSAT | attacker accuracy: 55.2% -> 65.9% | Helped |
Surprising findings
- DFME with 2M queries per class falls to near-random accuracy on EuroSAT and PASCAL, largely due to quantization and hard-label feedback breaking its gradient estimation.
- Stealix surpasses knowledge distillation with real victim data on PASCAL (40.0% vs 34.6%), because the synthetic pipeline issues more queries than the limited victim training set.
Limitations
- Relies on hard-label feedback; attacker accuracy improves noticeably with soft labels, and defenses against hard-label leakage are only partially effective.
- Performance depends on the attacker architecture and on the diffusion model's domain knowledge; limited medical knowledge reduces gains.
Experimental setup
Datasets: EuroSAT PASCAL VOC DomainNet CIFAR10 NSFW dataset (HuggingFace) PatchCamelyon RetinaMNIST
Benchmarks: EuroSAT PASCAL VOC CIFAR10 DomainNet NSFW classification
Metrics: attacker model accuracy prompt consistency Spearman correlation recall (diversity)
Baselines: DA-Fusion Real Guidance ASPKD Knockoff Nets DFME Knowledge distillation InstructBLIP prompts
Builds on: PEZ DA-Fusion Real Guidance ASPKD Knockoff Nets
Closest papers by meaning
- Efficient Model Stealing Defense with Noise Transition Matrix (CVPR 2024 · similarity 0.88)
- Exploring Query Efficient Data Generation Towards Data-Free Model Stealing in Hard Label Setting (AAAI 2025 · similarity 0.86)
- Fully Exploiting Every Real Sample: SuperPixel Sample Gradient Model Stealing (CVPR 2024 · similarity 0.86)
- Reconstruction Attacks on Machine Unlearning: Simple Models are Vulnerable (NeurIPS 2024 · similarity 0.86)
- Protecting Model Adaptation from Trojans in the Unlabeled Data (AAAI 2025 · similarity 0.85)
- Stealthy Imitation: Reward-guided Environment-free Policy Stealing (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.