Research map / Probabilistic and scientific modeling / Dynamical systems
Nonlinear dynamics of localization in neural receptive fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives early-time effective gradient-flow dynamics for a single ReLU neuron trained on a lengthscale discrimination task over natural-image-like data, showing that negative excess kurtosis of marginals drives localization while elliptical data does not.
Problem
Explaining how localized receptive fields emerge through learning in feedforward networks without explicit sparsity or efficient-coding constraints, and which data statistics drive this emergence.
Approach
Conditions on the input marginal X_i (assuming the preactivation is Gaussian conditional on X_i rather than unconditionally) to derive a closed-form early-time gradient flow for a single ReLU neuron; the nonlinear "amplifier" phi depends on marginal statistics, and its cubic Taylor term reveals that negative excess kurtosis yields super-linearity and localization.
Key results
- Localization (IPR near 1.0) occurs for negative excess kurtosis marginals (Ising, NLGP with large g), while positive excess kurtosis gives IPR near 0, consistent across 30 initializations.
- Integrated early-time dynamics reproduce the simulated peak position in 26 of 28 Ising initializations (93%).
- For elliptical data (e.g., t_40(nu=3)), the learned receptive fields are sinusoids with l2 fit errors of 9.77%, 3.75% and 4.14%, as predicted by the proposition.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use data with negative marginal excess kurtosis | Single-neuron model · NLGP(g) and Kur(k) | localization (IPR) | Helped |
| Use data with positive marginal excess kurtosis | Single-neuron model · NLGP(g) and Kur(k) | localization (IPR) | Hurt |
| Train on Ising data with marginal excess kurtosis -2 | Single-neuron model · Ising | receptive-field localization | Helped |
| Train on elliptical rather than weakly dependent data | Single-neuron model · Elliptical distributions | receptive-field localization | Hurt |
| Train on Student-t elliptical data with infinite preactivation kurtosis | Single-neuron model · t40(ν=3) | receptive-field localization | Hurt |
| Use Kur(10) rather than Kur(4) data | Soft committee machine with sigmoid activation · Kur(10) vs Kur(4) | receptive-field localization: excess kurtoses -0.93 vs 3.28 | Helped |
| Train the many-neuron model on Kur(30) rather than Kur(4) | Many-neuron model with learnable second-layer weights · Kur(30) vs Kur(4) | receptive-field localization: excess kurtoses -1.17 and 3.28 | Mixed |
| Use ICA rather than the neural network model | FastICA · Kur(3) | receptive-field localization: excess kurtosis 7.66 | Helped |
| Compare analytical dynamics with simulated training | Single-neuron model on Ising · Ising | peak-position match rate: 26 of 28 (93%) | Helped |
Surprising findings
- Elliptical distributions with heavy tails (even infinite kurtosis preactivations, e.g., Student-t with nu=3) fail to produce localization, showing preactivation kurtosis is not the right metric.
- ICA learns localized receptive fields on positive-excess-kurtosis data, contrary to the neural network models which require negative excess kurtosis.
Limitations
- Analysis is limited to early-time dynamics of a single bias-free ReLU neuron; the Gaussian-conditional assumption breaks down once localization emerges (Lindeberg violation).
- Does not capture orientation or phase selectivity, nor on-center-off-surround subfields seen in biological and artificial receptive fields.
- No exact necessary-and-sufficient threshold; the cubic Taylor approximation is only approximate and the claim is not proven for general kurtosis.
Experimental setup
Datasets: Ising model samples (Gibbs sampling) NLGP(g) nonlinear Gaussian process Kur(k) controllable-kurtosis model elliptical distributions (Student-t, ellipse surface)
Metrics: inverse participation ratio (IPR) l2 distance between fitted sinusoid and empirical receptive peak position match rate
Baselines: ICA (FastICA) sparse coding PCA Gaussian-equivalence gradient flow
Builds on: Ingrosso & Goldt 2022 data model of localization Gaussian equivalence property soft committee machine sparse coding ICA
Closest papers by meaning
- Neurons as Detectors of Coherent Sets in Sensory Dynamics (NeurIPS 2025 · similarity 0.88)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.87)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.87)
- Dendritic Localized Learning: Toward Biologically Plausible Algorithm (ICML 2025 · similarity 0.87)
- Shaping the distribution of neural responses with interneurons in a recurrent circuit model (NeurIPS 2024 · similarity 0.86)
- From Activation to Initialization: Scaling Insights for Optimizing Neural Fields (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.