Research map / Optimization / Implicit bias and generalization
Contribution of task-irrelevant stimuli to drift of neural representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that learning noise from task-irrelevant stimuli, which networks learn to ignore, drives long-term representational drift in Oja's rule, similarity matching, autoencoders and a two-layer network, with theory and simulations.
Problem
Representational drift after performance has converged is usually attributed to noise sources, but it is unclear which sources (especially sample-driven learning noise) drive drift across architectures and learning rules, and how drift depends on the data distribution.
Approach
In online learning, task-irrelevant stimuli that the network suppresses still produce nonzero stochastic updates; these fluctuations diffuse the solution along rotational tangent directions, so drift rate grows with variance and dimension of the task-irrelevant subspace. This differs qualitatively from isotropic Gaussian synaptic noise.
Key results
- Analytical diffusion for Oja and Similarity Matching scales as eta^3 lambda_perp^2 (m-1)(n-m)/8 for Gaussian data, matching simulations.
- On MNIST (n=20), drift rate vs output dimension rises to a maximum then falls to zero at m=n for Oja, SM and autoencoder.
- Synaptic Gaussian noise gives isotropic diffusion and monotonic dependence on output dimension, qualitatively different from learning-noise drift.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase task-irrelevant stimulus variance | Oja, Similarity Matching, autoencoder, and supervised two-layer networks · Gaussian data | drift rate | Helped |
| Increase task-irrelevant subspace dimension | Oja, Similarity Matching, and autoencoder · Gaussian data | drift rate: D_y \approx \frac{\eta^3\lambda_\perp^2}{8}(m-1)(n-m); D_h \approx \frac{\eta^3\lambda_{\perp}^2}{32} (p-1)(n-p) | Helped |
| Increase task-irrelevant stimulus variance | Nonlinear two-layer network · Ring position task | hidden-representation drift rate | Helped |
| Vary representation dimension | Oja, Similarity Matching, and autoencoder · MNIST projected onto top 20 principal components | drift rate | Mixed |
| Add Gaussian synaptic noise | Oja · Gaussian data | drift rate: D_{y} \approx \frac{\eta^3\lambda_\perp^2}{8}(m-1)(n-m) + \frac{1}{4} \eta\sigma_{syn}^2(m-1) | Helped |
| Replace learning-induced noise with additive Gaussian synaptic noise | Oja · Representation space | drift geometry: D_{sr} =\eta\sigma_{syn}^2/4 | Mixed |
| Increase synaptic noise strength | Oja · MNIST | drift rate versus output dimension | Mixed |
Surprising findings
- Drift rate vs output dimension is non-monotonic under learning noise, peaking at intermediate dimensions and vanishing when output dimension equals input dimension.
- Task-irrelevant stimuli, which are suppressed at the output, still cause drift of task-relevant representations.
Limitations
- Theory is derived for linear networks and small learning rates; nonlinear case is only shown numerically.
- Large learning rate effects (finite step size, edge of stability) are not captured.
- Predictions are not yet tested against experimental neural drift data.
Experimental setup
Datasets: MNIST synthetic Gaussian data
Benchmarks: MNIST
Metrics: drift rate (slope of autocorrelation decay) pairwise angular diffusion coefficient Grassmann distance cosine similarity
Baselines: Gaussian synaptic noise
Builds on: Oja's rule similarity matching network SGD diffusion approximation (stochastic differential equation)
Closest papers by meaning
- Disentangling Representations through Multi-task Learning (ICLR 2025 · similarity 0.85)
- Identifying Predictions That Influence the Future: Detecting Performative Concept Drift in Data Streams (AAAI 2025 · similarity 0.85)
- Feature Learning beyond the Lazy-Rich Dichotomy: Insights from Representational Geometry (ICML 2025 · similarity 0.84)
- Measuring Representational Shifts in Continual Learning: A Linear Transformation Perspective (ICML 2025 · similarity 0.84)
- Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron (NeurIPS 2024 · similarity 0.84)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.