Research map / Adversarial robustness and security / Attacks on text and graphs

Weakly Supervised Contrastive Adversarial Training for Learning Robust Features from Semi-supervised Data

CVPR 2025 · Adversarial robustness and security › Attacks on text and graphs · approach: Defenses against universal perturbations · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes WSCAT, a semi-supervised adversarial training method that uses a weakly supervised InfoNCE-style dynamic loss (positives chosen by current classifier predictions) to generate complete adversarial examples that disrupt non-robust feature–label correlations.

Problem

Existing adversarial training generates adversarial examples that perturb only some non-robust features (incomplete perturbation), leaving residual non-robust feature–label correlations; achieving complete perturbation is hard without explicit robust/non-robust labels and with scarce labeled data.

Approach

Formulate complete AE generation as an information-theoretic constraint minimizing mutual information between natural and adversarial embeddings, approximated by an InfoNCE lower bound whose positive set is defined by the current classifier's predictions (weak supervision), so no labels are needed for unlabeled data; pseudo-labels from Mean Teacher are used for the defense step.

Key results

• On CIFAR10, WSCAT reaches 80.93% natural and 59.40 harmonic-mean Mean accuracy, versus 49.21 for the best prior semi-supervised AT baseline (PUAT RST-style methods shown in table as RST 49.21 and PUAT 52.12).
• On CIFAR100, WSCAT achieves a harmonic mean of 27.43 versus 22.40 for PUAT.
• On ImageNet32-100, WSCAT achieves a harmonic mean of 10.59 versus 7.42 for PUAT and 5.71 for RST.

Measured effects reported in the paper

Surprising findings

• Using Mean Teacher pseudo-labels instead of standard-training pseudo-labels (WSCAT-std) gives a notably higher harmonic mean, showing pseudo-label quality matters for learning robust features.
• Standard (non-robust) training yields zero robust accuracy on all datasets, while MT-only semi-supervised training also gives near-zero robustness.

Limitations

• Complete perturbation is only approximated via an InfoNCE lower bound; the paper notes minimizing the bound does not guarantee minimizing MI.
• Theoretical results rely on assumptions and an approximation (Theorem 1 uses ≈), and the robust-feature guarantee is stated with probabilistic bounds.
• Requires tuning beta and lambda on a validation set, and the epoch time is roughly 5-13 minutes per epoch on a single RTX 4090.

Experimental setup

Datasets: CIFAR10 CIFAR100 ImageNet32-100

Benchmarks: CIFAR10 CIFAR100 ImageNet32-100

Metrics: natural accuracy robust accuracy (FGSM, PGD, CW, AutoAttack) harmonic mean

Baselines: Standard TRADES RLFAT DMAT WSCAT-sup MT RST PUAT

Builds on: TRADES InfoNCE Mean Teacher RST Ilyas et al. robust features

Closest papers by meaning

• Robust Pre-Training by Adversarial Contrastive Learning (NeurIPS 2020 · similarity 0.90)
• Adversarial Self-Supervised Contrastive Learning (NeurIPS 2020 · similarity 0.90)
• Unlabeled Data Improves Adversarial Robustness (NeurIPS 2019 · similarity 0.87)
• Adversarial Robustness with Semi-Infinite Constrained Learning (NeurIPS 2021 · similarity 0.87)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.87)
• Soften to Defend: Towards Adversarial Robustness via Self-Guided Label Refinement (CVPR 2024 · similarity 0.86)