Research map / Label-efficient and robust learning / Recognition and long-tail
Feature Contamination: Neural Networks Learn Uncorrelated Features and Fail to Generalize
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that SGD-trained two-layer ReLU networks learn uncorrelated background features alongside core features ("feature contamination"), causing OOD failure, supported by distillation experiments on CLIP teachers and deep-network evidence.
Problem
Understanding why neural networks fail to generalize out-of-distribution even when given good representations, and whether spurious correlations are the main cause.
Approach
Identifies feature contamination: due to asymmetric ReLU activation across classes, SGD gives non-zero gradient projections onto background features even when they are uncorrelated with the label, coupling core and background features in neuron pre-activations; linear networks do not exhibit this.
Key results
- Distilled models close only about half of the average effective robustness gap between standard and pre-trained models on ImageNet-based datasets, and do worse on iWildCam and DomainNet despite matching ID performance.
- Theory: for two-layer ReLU nets trained by SGD, training risk is o(1) while OOD risk is Θ̃(1) under the structured feature model with uncorrelated background features.
- Linear networks are shown to have background-feature weights of at most Õ(1/sqrt(d)), i.e., no feature contamination.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill a pretrained teacher's representations into a randomly initialized student | Student encoder · ImageNet-based distribution-shift datasets | effective robustness | Helped |
| Use a distilled student instead of the pretrained teacher | Pretrained encoders · ImageNet-based distribution-shift datasets | OOD performance | Hurt |
| Replace ReLU with the identity function (ablation) | Two-layer ReLU network trained by SGD · Structured binary classification with OOD shifts in background features | background-feature learning | Helped |
| Learn uncorrelated background features alongside core features | SGD-trained two-layer ReLU network · Structured binary classification with OOD shifts in background features | OOD risk | Hurt |
| Train a randomly initialized student to fit representations from an OOD-generalizable teacher | Student encoder · Distribution-shift benchmarks | OOD performance | Hurt |
Surprising findings
- Even when a student network explicitly fits representations from a robust teacher, OOD performance still lags far behind the pre-trained teacher.
- Feature contamination occurs even when background features are uncorrelated with the label and under weight decay, so spurious correlations are not required.
Limitations
- Theory restricted to binary classification, two-layer ReLU networks with fixed output layer, and orthogonal features.
- Pre-training connection is only a conjecture supported by preliminary evidence.
Experimental setup
Datasets: ImageNet-based natural distribution shift datasets WILDS iWildCam WILDS DomainNet CIFAR-10 (modified with uncorrelated background color)
Benchmarks: ImageNet-based natural distribution shift datasets iWildCam DomainNet
Metrics: effective robustness ID accuracy OOD accuracy OOD risk neuron selectivity
Baselines: ERM CLIP pre-trained encoders standard ImageNet-trained models
Builds on: representation distillation effective robustness winning lottery tickets Grad-CAM
Closest papers by meaning
- Uncertainty Modeling for Out-of-Distribution Generalization (ICLR 2022 · similarity 0.88)
- Feature Averaging: An Implicit Bias of Gradient Descent Leading to Non-Robustness in Neural Networks (ICLR 2025 · similarity 0.87)
- On the Clean Generalization and Robust Overfitting in Adversarial Training from Two Theoretical Views: Representation Complexity and Training Dynamics (ICML 2025 · similarity 0.87)
- The Implicit Bias of Structured State Space Models Can Be Poisoned With Clean Labels (NeurIPS 2025 · similarity 0.87)
- What Variables Affect Out-of-Distribution Generalization in Pretrained Models? (NeurIPS 2024 · similarity 0.87)
- Data Distributional Properties As Inductive Bias for Systematic Generalization (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.