Research map / Label-efficient and robust learning / Recognition and long-tail
Red Teaming Deep Neural Networks with Feature Synthesis Tools
Stephen Casper, Tong Bu, Yuxiao Li, Jiawei Li, Kevin Zhang, Kaivalya Hariharan, Dylan Hadfield-Menell
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use feature synthesis instead of feature attribution/saliency for trojan discovery | Interpretability tools · 12-trojan benchmark | human trigger identification | Improved |
| Use robust feature-level adversaries or SNAFUE rather than other individual synthesis tools | Feature synthesis tools · 12-trojan benchmark | human trigger identification | Improved |
| Combine visualizations from multiple tools rather than relying on one tool | Interpretability toolbox · 12-trojan benchmark | human trigger identification | Improved |
| Visualize target neurons instead of multiple associated inner neurons (ablation) | Feature visualization · Trojan trigger identification | human trigger identification | Improved |
| Use occlusion rather than an edge detector baseline | Feature attribution/saliency · Patch trojans on ResNet50 and VGG19 | pixel-wise Pearson correlation | Improved |
| Use Fourier-space feature visualization or TABOR | Feature synthesis tools · 12-trojan benchmark | human trigger identification | No clear effect |
| Use feature synthesis methods to rediscover style-transfer trojans | Feature synthesis tools · Style trojans | human trigger identification | Hurt |
| Use human evaluators instead of CLIP to judge visualizations | Interpretability tool evaluation · 12-trojan benchmark | correct trigger identification | Improved |
| Use SNAFUE for copy/paste attacks between dissimilar source and target classes | SNAFUE · Clothing-to-traffic ImageNet attacks | natural patch attack success: Only one out of the 1,000 total natural patches ... succeeded for at least 50% of source class images; 54% ... were succ | Mixed |
| Use SNAFUE to recover an implanted trigger included among candidate patches | SNAFUE · Four patch-trojan cases | trigger recovered in top 10: 2 of the 4 cases | Mixed |
Closest papers by meaning
- A Multimodal Automated Interpretability Agent (ICML 2024 · similarity 0.86)
- Measuring Per-Unit Interpretability at Scale Without Humans (NeurIPS 2024 · similarity 0.86)
- Neurons to Words: A Novel Method for Automated Neural Network Interpretability and Alignment (AAAI 2025 · similarity 0.85)
- Everything, Everywhere, All at Once: Is Mechanistic Interpretability Identifiable? (ICLR 2025 · similarity 0.85)
- From Black-box to Causal-box: Towards Building More Interpretable Models (NeurIPS 2025 · similarity 0.84)
- Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
