Research map / Optimization / Implicit bias and generalization

Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks

ICLR 2025 · Optimization › Implicit bias and generalization · approach: Implicit bias in linear networks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proves that gradient descent on two-layer ReLU networks trained on multi-cluster orthogonal data converges to feature-averaging solutions that are non-robust, and that fine-grained cluster-level supervision yields decoupled features with optimal robustness.

Problem

Explaining why gradient-descent-trained neural networks are non-robust to adversarial l2 perturbations even when robust solutions exist, given multiple discriminative features per class.

Approach

Gradient descent on two-layer ReLU networks with small initialization provably learns, for each neuron, a weighted average of the cluster-center features of its class (feature averaging), which is vulnerable to perturbations along the negative averaged direction; training with cluster-level (multi-class) labels instead makes each neuron align with a single cluster feature and yields optimal robust

Key results

• Theorem: under the stated hyperparameters, GD converges to feature-averaging weights (||w - lambda * avg of cluster centers|| <= o(d^{-1/2})) with near-perfect clean accuracy, and the delta-robust accuracy at delta = Omega(sqrt(d/k)) is nearly zero.
• Multi-class (cluster-label) training yields decoupled weights aligned with single cluster features and binary robustness at radius O(sqrt(d)), which is optimal up to constants.
• Adversarial training on synthetic data gives less pronounced decoupling and somewhat lower robustness than fine-grained supervision.

Measured effects reported in the paper

Surprising findings

• A network with perfect clean accuracy and very low multi-class loss can still be non-robust, so fine-grained supervision alone does not guarantee robustness without the implicit bias.

Limitations

• Theory restricted to two-layer ReLU networks with fixed second layer, orthogonal equinorm cluster centers, and small initialization/learning rate
• Requires large d relative to k and large n polynomial in k

Experimental setup

Datasets: synthetic multi-cluster Gaussian data MNIST CIFAR-10

Benchmarks: MNIST CIFAR-10 synthetic multi-cluster data

Metrics: clean accuracy l2 robust accuracy cosine similarity between cluster features and weights

Baselines: binary-label training 10-class fine-grained training adversarial training (PGD)

Builds on: Li et al. 2022 (multi-cluster data / feature learning) Sanyal et al. 2020 (implicit bias and robustness) Frei et al. 2023 (benign overfitting auto-balance) Allen-Zhu & Li feature learning framework

Closest papers by meaning

• The Double-Edged Sword of Implicit Bias: Generalization vs. Robustness in ReLU Networks (NeurIPS 2023 · similarity 0.90)
• The Implicit Bias of Gradient Descent toward Collaboration between Layers: A Dynamic Analysis of Multilayer Perceptions (NeurIPS 2024 · similarity 0.90)
• Gradient Methods Provably Converge to Non-Robust Networks (NeurIPS 2022 · similarity 0.90)
• Adversarial Training Can Provably Improve Robustness: Theoretical Analysis of Feature Learning Process Under Structured Data (ICLR 2025 · similarity 0.89)
• Can Implicit Bias Imply Adversarial Robustness? (ICML 2024 · similarity 0.89)
• The Price of Implicit Bias in Adversarially Robust Generalization (NeurIPS 2024 · similarity 0.88)