Research map / Adversarial robustness and security / Certified and empirical robustness
Two Heads are Actually Better than One: Towards Better Adversarial Robustness via Transduction and Rejection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TLDR, a transductive learning defense with rejection that adversarially trains on train and test inputs and rejects inputs near the decision boundary, using a practical empirical version of Tramèr's classifier-to-detector reduction plus an adaptive GMSA-based attack.
Problem
Practical robust classification under bounded perturbations that combines transduction and rejection; prior transduction+rejection work (Goldwasser et al., URejectron) gave weak robustness on deep learning under strong attacks like GMSA.
Approach
Applies Tramèr's classifier-to-detector reduction in the transductive setting with bounded perturbations, yielding improved sample complexity and a weaker data assumption (OPT at U^{2/3}); an empirical version rejects test inputs whose prediction changes within a rejection radius found by PGD, combined with transductive adversarial training and a rejection-aware adaptive attack.
Key results
- TLDR reaches 81.6% robust accuracy on CIFAR-10 with WideResNet-28-10 (l_inf 8/255, GMSA with rejection-aware loss), versus 71.1% for the prior state of the art.
- On CIFAR-100, TLDR reaches 57.9% robust accuracy versus the strongest existing baseline of 42.7%.
- On MNIST, TLDR achieves 0.972 robust accuracy versus 0.721 for URejectron.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine transduction with rejection | Adversarial defense on MNIST · MNIST | robust accuracy: 0.968 -> 0.972 | Helped |
| combine transduction with rejection | Adversarial defense on CIFAR-10 · CIFAR-10 | robust accuracy: 0.634 -> 0.739 | Helped |
| use TLDR instead of the strongest listed inductive baseline | WideResNet-28-10 adversarial defense · CIFAR-10 | robust accuracy: 0.711 -> 0.816 | Helped |
| use TLDR instead of the strongest listed inductive baseline | WideResNet-28-10 adversarial defense · CIFAR-100 | robust accuracy: 0.427 -> 0.579 | Helped |
| replace the cross-entropy attack loss with the rejection-aware LREJ loss (ablation) | GMSA attack against TLDR · MNIST | robust accuracy: 0.988 -> 0.972 | Hurt |
| replace the cross-entropy attack loss with the rejection-aware LREJ loss (ablation) | GMSA attack against TLDR · CIFAR-10 | robust accuracy: 0.853 -> 0.739 | Hurt |
| use the LREJ attack loss instead of cross-entropy (ablation) | PGD attack on a fixed adversarially trained model with rejection · CIFAR-10 | robust accuracy: 0.524 -> 0.458 | Hurt |
Surprising findings
- Rejection-aware GMSA is markedly stronger than standard GMSA with cross-entropy loss against TLDR (0.739 vs 0.853 on CIFAR-10 with ResNet-20).
- The rejection-only AT defense with budget 4/255 at training closely matches the 8/255 attack result, showing the empirical transformation approaches the theoretical half-budget bound.
Limitations
- Training must be repeated for each new batch of test samples, limiting latency-sensitive use.
- The theoretical guarantee does not hold for the empirical transformation; the reduction is computationally inefficient in its original form and the practical version lacks guarantees.
- Attacking transductive selective classifiers is very computationally expensive, so evaluation is limited.
Experimental setup
Datasets: MNIST CIFAR-10 CIFAR-100
Benchmarks: MNIST CIFAR-10 CIFAR-100
Metrics: robust accuracy rejection rate
Baselines: AT (PGD adversarial training) AT with rejection RMC DANN TADV URejectron (Rejectron)
Builds on: Tramèr classifier-to-detector reduction GMSA TADV Goldwasser et al. transductive rejection (URejectron)
Closest papers by meaning
- Towards Evaluating the Robustness of Neural Networks Learned by Transduction (ICLR 2022 · similarity 0.91)
- Adversarial Training on Purification (AToP): Advancing Both Robustness and Generalization (ICLR 2024 · similarity 0.86)
- Boosting Adversarial Transferability through Augmentation in Hypothesis Space (CVPR 2025 · similarity 0.85)
- Certified Causal Defense with Generalizable Robustness (AAAI 2025 · similarity 0.85)
- Efficient Robustness Evaluation via Constraint Relaxation (AAAI 2025 · similarity 0.85)
- PubDef: Defending Against Transfer Attacks From Public Models (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.