Research map / Adversarial robustness and security / Certified and empirical robustness

Gain from Neighbors: Boosting Model Robustness in the Wild via Adversarial Perturbations Toward Neighboring Classes

CVPR 2025 · Adversarial robustness and security › Certified and empirical robustness · approach: Robust overfitting and regularization · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes GFN, which fine-tunes pretrained ViT/FAN models on gradient perturbations targeting semantically neighboring classes plus an inter-class distance-weighted cross-entropy, to improve robustness to ImageNet-C and other distribution shifts.

Problem

Pretrained vision models degrade sharply under distribution shifts such as corruptions, style changes and adversarial-like perturbations; existing augmentation, adversarial training and architecture changes are costly or lack insight into which shifts matter.

Approach

Observes that misclassifications under distribution shift cluster among semantically similar classes, and generates targeted gradient perturbations pushing inputs toward top-k neighboring classes (instead of arbitrary adversarial directions), then fine-tunes with a loss weighted by inter-class similarity rank so that similar classes are more separated.

Key results

• On ImageNet-C, GFN with ViT-B reaches mCE 41.4 vs 43.7 for TAPADL (FAN), and GFN with FAN reaches 39.8.
• Ours (ViT-B) raises clean ImageNet top-1 from 76.3% to 79.9% over the ViT-B baseline.
• Ablation on ViT-B: neighboring-class perturbation alone gives IN-C mCE 43.5 vs 52.2 baseline; adding the weighted loss gives 41.4.

Measured effects reported in the paper

Surprising findings

• Fine-tuning for 10 epochs on all ImageNet images gave similar results to using only a 5000-image subset, suggesting the method needs little data.

Limitations

• Gains are evaluated mainly on ViT-B and FAN backbones and on a limited set of benchmarks.
• Hyperparameters k and α were selected by a quick test on motion blur images, which overlaps with evaluation domains.

Experimental setup

Datasets: ImageNet-1k COCO COCO-C Foggy-Cityscapes RTTS ExDark

Benchmarks: ImageNet ImageNet-C ImageNet-P ImageNet-A ImageNet-R COCO-C Foggy-Cityscapes RTTS ExDark

Metrics: top-1 accuracy mCE mFR mAP

Baselines: ViT-B DeiT-B Swin-B RVT FAN RSPC TAPADL

Builds on: adversarial training (FGSM-style gradient perturbation) cross-entropy loss ImageNet-C benchmark

Closest papers by meaning

• RobustGenBench: A Benchmark for Robust Generalization to Adversarial and Common Perturbations, with Applications to Vision and Vision-Enabled Large Language Models (NeurIPS 2026 · similarity 0.87)
• Adversarially Robust Few-shot Learning via Parameter Co-distillation of Similarity and Class Concept Learners (CVPR 2024 · similarity 0.87)
• Distilling Out-of-Distribution Robustness from Vision-Language Foundation Models (NeurIPS 2023 · similarity 0.87)
• Hierarchically Robust Zero-shot Vision-language Models (CVPR 2026 · similarity 0.86)
• The Geometry of Robustness: Optimizing Loss Landscape Curvature and Feature Manifold Alignment for Robust Finetuning of Vision-Language Models (CVPR 2026 · similarity 0.86)
• LORE: Lagrangian-Optimized Robust Embeddings for Visual Encoders (NeurIPS 2025 · similarity 0.85)