Research map / Adversarial robustness and security / Attacks on text and graphs
Scanning Trojaned Models Using Out-of-Distribution Samples
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TRODO, a trojan scanner that adversarially shifts OOD samples toward in-distribution regions and uses the increase in MSP ID-score as a signature, working with or without clean training data.
Problem
Detecting trojaned (backdoored) image classifiers in a general way, agnostic to attack type and label mapping, including classifiers trained adversarially, and without requiring training data.
Approach
Trojaned classifiers have blind spots where OOD samples are confidently classified as ID. TRODO runs a PGD attack that raises the MSP ID-score of hard-augmented OOD samples and uses the mean ID-score gap between clean and perturbed samples, with a threshold from a clean surrogate, as the trojan signature.
Key results
- TRODO achieves 90.7% average accuracy on standard-trained classifiers and 87.3% on adversarially trained ones in the designed benchmark (All-to-One).
- TRODO-Zero, which uses no training data, achieves 79.4% average accuracy in All-to-One settings.
- On TrojAI Round0, TRODO reaches 86.2% accuracy with 152.4 s average scanning time, versus K-ARM 91.3% at 262.1 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TRODO rather than the strongest baseline on average | All-to-One scanning · All-to-One, Avg. | accuracy (ACC): 79.3 -> 90.7 | Helped |
| use TRODO rather than the strongest baseline on average | All-to-One scanning · All-to-One, Avg. | accuracy (ACC*): 72.3 -> 87.3 | Helped |
| use TRODO rather than the strongest baseline on average | All-to-All scanning · All-to-All, Avg. | accuracy (ACC): 76.1 -> 87.5 | Helped |
| replace available benign training samples with Tiny ImageNet (ablation) | TRODO · All-to-One, Avg. | accuracy (ACC): 90.7 -> 79.4 | Hurt |
| replace Tiny ImageNet validation/OOD data with another validation set (ablation) | TRODO · CIFAR10 | accuracy: 78.7, 82.6, 96.8, 91.0 | Mixed |
| vary boundary confidence level gamma from 0.2 to 0.8 (ablation) | TRODO · MNIST | accuracy: 81.0, 74.8, 89.1, 91.2, 85.2, 75.6, 81.8 | Mixed |
| train the classifier using adaptive loss 1 instead of the default loss | TRODO · All-to-One, Avg. | accuracy (ACC): 90.7 -> 85.3 | Hurt |
| train the classifier using adaptive loss 2 instead of the default loss | TRODO · All-to-All, Avg. | accuracy (ACC): 87.5 -> 75.9 | Hurt |
Surprising findings
- Trojaned classifiers confidently classify even far-OOD samples as ID once the trigger is applied, consistent with benign overfitting.
- Previous scanners drop sharply on adversarially trained trojaned classifiers, while TRODO remains robust.
Limitations
- Performance on TrojAI is below K-ARM on rounds 0-1 in accuracy.
- Relies on choice of OOD validation dataset; near-OOD choice matters (Table on validation datasets).
- Adaptive attacks reduce accuracy (e.g., to 74-85% ACC*), though TRODO remains reasonably robust.
Experimental setup
Datasets: CIFAR10 CIFAR100 GTSRB PubFig MNIST Tiny ImageNet STL-10 FMNIST SVHN TrojAI
Benchmarks: Designed benchmark (MNIST, CIFAR10, GTSRB, CIFAR100, PubFig; All-to-One and All-to-All; 8 attacks) TrojAI rounds 0-4 and 11
Metrics: accuracy ACC* (adversarially trained classifiers) scanning time
Baselines: NC ABS PT-RED TABOR K-ARM MNTD FreeEagle MM-BD UMD
Builds on: PGD DeepFool MSP baseline for OOD detection Neural Cleanse MM-BD UMD
Closest papers by meaning
- Attack To Defend: Exploiting Adversarial Attacks for Detecting Poisoned Models (CVPR 2024 · similarity 0.85)
- Protecting Model Adaptation from Trojans in the Unlabeled Data (AAAI 2025 · similarity 0.84)
- Deep-TROJ: An Inference Stage Trojan Insertion Algorithm through Efficient Weight Replacement Attack (CVPR 2024 · similarity 0.83)
- Splitting & Integrating: Out-of-Distribution Detection via Adversarial Gradient Attribution (ICML 2025 · similarity 0.83)
- PSBD: Prediction Shift Uncertainty Unlocks Backdoor Detection (CVPR 2025 · similarity 0.83)
- RPP: A Certified Poisoned-Sample Detection Framework for Backdoor Attacks under Dataset Imbalance (NeurIPS 2026 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.