Research map / Optimization / Implicit bias and generalization
Stochastic Gradient Descent-Induced Drift of Representation in a Two-Layer Neural Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase stimulus presentation frequency (frequent stimulus with probability alpha) | representation of frequent stimulus under SGD online learning · two-layer linear autoencoder, analytic and simulations | diffusion coefficient (drift rate) | Helped |
| Increase stimulus presentation frequency (frequent stimulus) | representation fluctuation of frequent stimulus · two-layer linear autoencoder, analytic | fluctuation variance of representation norm | Helped |
| Add SGD sampling noise on the manifold (no external noise) with L2 regularization | representation drift in two-layer linear network · Gaussian stimuli | diffusion coefficient D_s | No clear effect |
Closest papers by meaning
- Contribution of task-irrelevant stimuli to drift of neural representations (NeurIPS 2025 · similarity 0.93)
- The Butterfly Effect: Neural Network Training Trajectories Are Highly Sensitive to Initial Conditions (ICML 2025 · similarity 0.85)
- Learning Reveals Invisible Structure in Low-Rank RNNs (NeurIPS 2026 · similarity 0.85)
- Modeling Latent Neural Dynamics with Gaussian Process Switching Linear Dynamical Systems (NeurIPS 2024 · similarity 0.85)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.85)
- Dynamical phases of short-term memory mechanisms in RNNs (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.