Research map / Probabilistic and scientific modeling / Dynamical systems
Dynamics of Supervised and Reinforcement Learning in the Non-Linear Perceptron
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives stochastic-process flow equations for the mean and covariance of weights in a nonlinear perceptron trained with supervised (logistic regression) or REINFORCE reinforcement learning on Gaussian binary classification, and studies how input noise affects learning speed and continual-learning forgetting, verified on MNIST digits 0/1.
Problem
Learning dynamics of nonlinear perceptrons for binary classification have been analyzed only under linearly separable student-teacher or linearized-output assumptions, leaving the roles of nonlinearity, input-data distribution and learning rule unclear.
Approach
Uses a Fokker-Planck-style stochastic-process approach, deriving the update noise statistics from the per-sample stochastic update, to obtain closed-form flow equations for the mean and covariance of weights in a nonlinear (error-function sigmoid) perceptron, comparing SL and RL learning rules under Gaussian input noise.
Key results
- For supervised learning, higher isotropic input noise speeds learning (alignment reaches 80% faster), while for RL the effect is indeterminate depending on parameters.
- For both SL and RL, noise shifted along the decoding direction slows learning while noise orthogonal to it speeds learning.
- Theoretical learning curves closely match SGD simulations on MNIST (0 vs 1) test accuracy; forgetting curves decay roughly exponentially with number of tasks and higher noise causes faster forgetting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase isotropic input noise | SL nonlinear perceptron · Binary Gaussian classification | learning speed | Helped |
| Increase isotropic input noise | RL nonlinear perceptron · Binary Gaussian classification | learning speed | Mixed |
| Shift fixed total input noise toward the decoding direction | SL and RL nonlinear perceptrons · Binary Gaussian classification | learning speed | Hurt |
| Shift fixed total input noise toward directions orthogonal to the decoding direction | SL and RL nonlinear perceptrons · Binary Gaussian classification | learning speed | Helped |
| Increase input noise | Nonlinear perceptron trained continually on tasks · Continual learning | forgetting rate | Hurt |
| Change the learning algorithm between SL and RL | Nonlinear perceptron trained continually on tasks · Continual learning | forgetting rate | No clear effect |
| Use the derived flow equations to predict training dynamics | Nonlinear perceptron · MNIST 0-vs-1 | test classification accuracy | No clear effect |
Surprising findings
- Higher input noise leads to faster learning for supervised learning, counterintuitively.
- Only a few tasks can be remembered by the perceptron, far fewer than the O(N) patterns recalled by binary perceptron models in prior work.
Limitations
- Assumes input distributions are multivariate Gaussians, which may not hold for real data.
- Neglects higher-order terms in weight deviation and in learning rate, limiting treatment of finite learning rates and instabilities.
- Replaces logistic sigmoid with a shifted error-function sigmoid for analytic tractability.
Experimental setup
Datasets: MNIST
Benchmarks: MNIST
Metrics: test classification accuracy alignment of weights with mean direction weight covariance trace forgetting curve
Baselines: direct simulation of SGD
Builds on: REINFORCE Fokker-Planck equation online logistic regression
Closest papers by meaning
- Learning dynamics in linear recurrent neural networks (ICML 2025 · similarity 0.89)
- Learning Dynamics of RNNs in Closed-Loop Environments (NeurIPS 2025 · similarity 0.88)
- Dynamical phases of short-term memory mechanisms in RNNs (ICML 2025 · similarity 0.87)
- Learning the Plasticity: Plasticity-Driven Learning Framework in Spiking Neural Networks (NeurIPS 2025 · similarity 0.86)
- Feedback control guides credit assignment in recurrent neural networks (NeurIPS 2024 · similarity 0.86)
- Measuring and Controlling Solution Degeneracy across Task-Trained Recurrent Neural Networks (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.