Research map / Adversarial robustness and security / Certified and empirical robustness

Adversarially Robust Few-shot Learning via Parameter Co-distillation of Similarity and Class Concept Learners

CVPR 2024 · Adversarial robustness and security › Certified and empirical robustness · approach: Adversarial training with feature consistency · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes RESISTANCE, which co-distills parameters of a visual similarity learner and a class concept learner into a unified embedding model to improve adversarially robust few-shot learning, with cross-branch class-wise global adversarial initialization and branch robustness harmonization.

Problem

Few-shot learners are vulnerable to adversarial attacks, and existing robust FSL methods rely on either visual similarity learning or class concept learning, which have complementary but unexploited robustness profiles.

Approach

Co-distill parameters from a similarity learner (AQ-style prototype loss) and a class concept learner (TRADES-style classifier) via EMA into a unified embedding, periodically redistribute back to both branches, seed PGD adversaries with cross-branch class-wise global perturbations, and reweight branch learning rates by relative adversarial robustness.

Key results

• On CIFAR-FS 5-way 5-shot with ResNet-12, RESISTANCE reaches 74.83% clean and 58.76% AutoAttack robust accuracy, versus 9% clean-accuracy gain over prior methods as reported.
• On Mini-ImageNet 5-way 1-shot with ResNet-12, RESISTANCE achieves 50.28% clean and 33.71% AutoAttack robust accuracy, outperforming DFSL (47.16% / 31.62%).
• Single-step adversary variant of RESISTANCE (GradAlign) retains robustness close to its multi-step counterpart, e.g., 31.33% vs 33.71% AA on Mini-ImageNet 1-shot, with lower training time.

Measured effects reported in the paper

Surprising findings

• Similarity and class concept learners show complementary robustness across attack strengths: similarity learners are more robust at small radii while concept learners are more robust at large radii.

Limitations

• Relies on multi-step PGD adversary generation by default, which is computationally expensive; single-step variants lose some robustness.
• Adds hyperparameters (EMA decay, balancing factor gamma, steepness tau, redistribution epoch and frequency) that require tuning.

Experimental setup

Datasets: Mini-ImageNet CIFAR-FS FC100

Benchmarks: Mini-ImageNet CIFAR-FS FC100

Metrics: clean accuracy robust accuracy (PGD-20) robust accuracy (CW) robust accuracy (AutoAttack)

Baselines: AQ R-MAML ST GR DFSL TROBA N-FGSM RS-FGSM GradAlign

Builds on: Adversarial Querying (AQ) TRADES Goldblum et al. adversarial querying universal adversarial perturbations exponential moving average

Closest papers by meaning

• Adversarially Robust Few-Shot Learning: A Meta-Learning Approach (NeurIPS 2020 · similarity 0.90)
• Gain from Neighbors: Boosting Model Robustness in the Wild via Adversarial Perturbations Toward Neighboring Classes (CVPR 2025 · similarity 0.87)
• Improving Zero-Shot Adversarial Robustness in Vision-Language Models by Closed-form Alignment of Adversarial Path Simplices (ICML 2025 · similarity 0.87)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.87)
• Learning Better Certified Models from Empirically-Robust Teachers (NeurIPS 2026 · similarity 0.87)
• Few-Shot Adversarial Prompt Learning on Vision-Language Models (NeurIPS 2024 · similarity 0.87)