Research map / Label-efficient and robust learning / Self-supervised and contrastive
Self-Supervised Adversarial Training via Diverse Augmented Queries and Self-Supervised Double Perturbation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DAQ-SDP, a framework-agnostic self-supervised adversarial training method that uses diversely augmented clean queries with pairwise BatchNorm and self-supervised weight perturbation to improve robustness and clean accuracy.
Problem
Self-supervised adversarial training (SAT) suffers from a large robust generalization gap and clean accuracy degradation, and its formulation and connection to supervised AT are under-explored.
Approach
Uses diversely augmented (TrivialAugment) clean features as pseudo-supervision with an Aug-Adv pairwise-BatchNorm student, and applies adversarial weight perturbation to the SSL similarity loss in a later stage of two-stage SAT, challenging the view that strong augmentations hurt SAT robustness.
Key results
- On CIFAR-100 with ResNet-34 and SimCLR, DAQ-SDP reaches 53.54% clean, 33.09% PGD and 23.42% AutoAttack, versus 51.44/30.68/21.31 for DecoupledACL.
- On CIFAR-10 with ResNet-34 and SimCLR, DAQ-SDP reaches 84.57% clean, 58.57% PGD and 49.22% AutoAttack.
- Ablation on CIFAR-100: diverse augmented query adds +1.23 clean and +1.43 PGD; adding weight self-perturbation brings total gains to +2.10 clean and +2.41 PGD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply DAQ-SDP instead of DecoupledACL | ResNet-34 with SimCLR · CIFAR-10 | AutoAttack accuracy: 47.99 -> 49.22 | Helped |
| apply DAQ-SDP instead of DecoupledACL | ResNet-34 with SimCLR · CIFAR-100 | AutoAttack accuracy: 21.31 -> 23.42 | Helped |
| apply DAQ-SDP instead of DecoupledACL | ResNet-34 with SimSiam · CIFAR-10 | PGD accuracy: 57.17 -> 58.53 | Helped |
| apply DAQ-SDP instead of DecoupledACL | ResNet-34 with BYOL · CIFAR-10 | AutoAttack accuracy: 47.67 -> 49.07 | Helped |
| add diverse augmented queries (ablation) | SAT on ResNet-34 with SimCLR · CIFAR-100 | PGD accuracy: 30.68 -> 32.11 | Helped |
| add diverse augmented queries (ablation) | SAT on ResNet-34 with SimCLR · CIFAR-100 | Clean accuracy: 51.44 -> 52.67 | Helped |
| add weight self-perturbation (ablation) | SAT on ResNet-34 with SimCLR · CIFAR-100 | PGD accuracy: 30.68 -> 32.37 | Helped |
| add weight self-perturbation (ablation) | SAT on ResNet-34 with SimCLR · CIFAR-100 | Clean accuracy: 51.44 -> 51.56 | No clear effect |
| use Aug-Adv pairwise-BatchNorm rather than single-BatchNorm (ablation) | DAQ on ResNet-34 with SimCLR · CIFAR-100 | PGD accuracy: 31.56 -> 33.09 | Helped |
| use Aug-Adv pairwise-BatchNorm rather than single-BatchNorm (ablation) | DAQ on ResNet-34 with SimCLR · CIFAR-100 | Clean accuracy: 52.27 -> 53.54 | Helped |
Surprising findings
- Contrary to prior SAT work, strong and diverse augmentations (TrivialAugment) improve robustness when the model has sufficient capacity.
- Improvements are smaller on ResNet-18, which the authors attribute to insufficient model capacity for richer augmented data.
Limitations
- Improvements are less significant on the smaller ResNet-18, implying dependence on model capacity.
- Added computation from extra adversarial streams and weight perturbation.
- Evaluated only on CIFAR-10/100 and STL-10 transfer with ResNet backbones.
Experimental setup
Datasets: CIFAR-10 CIFAR-100
Benchmarks: CIFAR-10 CIFAR-100 STL-10
Metrics: clean accuracy PGD-20 robust accuracy AutoAttack accuracy
Baselines: DynACL+AIR TARO DecoupledACL
Builds on: SimCLR SimSiam BYOL DecoupledACL TrivialAugment adversarial weight perturbation
Closest papers by meaning
- 3SAT: A Simple Self-Supervised Adversarial Training Framework (AAAI 2025 · similarity 0.92)
- Rethinking the Effect of Data Augmentation in Adversarial Contrastive Learning (ICLR 2023 · similarity 0.90)
- Adversarial Robustness: From Self-Supervised Pre-Training to Fine-Tuning (CVPR 2020 · similarity 0.89)
- Contrastive Self-Supervised Learning Leads to Higher Adversarial Susceptibility (AAAI 2023 · similarity 0.88)
- Annealing Self-Distillation Rectification Improves Adversarial Training (ICLR 2024 · similarity 0.87)
- Long-tailed Adversarial Training with Self-Distillation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.