Research map / Probabilistic and scientific modeling / Dynamical systems
Shaping the distribution of neural responses with interneurons in a recurrent circuit model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a normative recurrent circuit model in which primary neurons interact with interneurons whose gains and activation functions adapt, derived from an optimal transport objective, and demonstrates it Gaussianizes natural image filter responses while reducing mutual information.
Problem
How neural circuits with local interneurons nonlinearly transform inputs into efficient response distributions (e.g., redundancy reduction toward a spherical Gaussian), given that prior normative models are largely linear.
Approach
Derives an online optimal-transport-based algorithm, mapped to a circuit where primary neurons are recurrently connected to interneurons; Hebbian plasticity learns projection directions while interneuron gains and activation functions adapt to nonlinearly reshape marginal responses toward a target Gaussian.
Key results
- With K=3 interneurons, the circuit significantly reduces mutual information between response pairs for spatial offsets below d=32, whereas ZCA whitening gives little reduction beyond d=2.
- In the scalar case, learned interneuron activations f(θ,z)=a(θ)z+b(θ)sign(z)|z|^θ closely approximate the optimal Gaussianizing transform for generalized Gaussian marginals of natural image filter responses.
- K=3 interneurons achieve about the same mutual-information reduction as K=4 and a much larger reduction than K=2.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply the nonlinear recurrent circuit transformation | Kodak local filter-response pairs · Spatial offsets less than d=32 | mutual information | Helped |
| Increase interneurons from K=2 to K=3 (ablation) | Proposed circuit · Kodak filter-response pairs, spatial offsets less than d=32 | mutual information | Helped |
| Increase interneurons from K=3 to K=4 (ablation) | Proposed circuit · Kodak filter-response pairs, spatial offsets less than d=32 | mutual information | No clear effect |
| Apply linear ZCA whitening | Local filter-response pairs · Kodak filter-response pairs, offsets other than d=2 | mutual information | No clear effect |
| Apply linear ZCA whitening | Local filter-response pairs · Kodak filter-response pairs | mutual information | Mixed |
| Apply the circuit transformation | Proposed circuit · Kodak filter-response pairs, spatial offsets greater than d=32 | mutual information | No clear effect |
| Use the learned nonlinear transformation | Proposed circuit · Scalar local filter responses from Kodak images | Gaussianization | Helped |
| Use the learned activation family on more sparse, peaked input distributions | Proposed circuit · Scalar local filter responses with smaller fitted shape parameter β | Gaussianization | Hurt |
Surprising findings
- ZCA whitening can slightly increase mutual information between coordinates for some spatial offsets, though the authors suggest this may be a discretization artifact.
- The circuit performs worse for sparse, peaked marginals (small shape parameter β), showing a characteristic dip in responses near zero.
Limitations
- Only two-dimensional inputs are tested, so scaling to high-dimensional natural signals is unvalidated.
- Primary neurons are linear and synapses are not sign-constrained, violating Dale's law.
- Circuit fails to fully Gaussianize marginals with peaked distributions near zero (small β).
Experimental setup
Datasets: Kodak image dataset
Metrics: mutual information between response coordinates distance to spherical Gaussian (histograms, contour plots)
Baselines: ZCA whitening
Builds on: adaptive whitening circuits projection pursuit sliced Wasserstein distance
Closest papers by meaning
- The Bayesian sampling in a canonical recurrent circuit with a diversity of inhibitory interneurons (NeurIPS 2024 · similarity 0.88)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.88)
- How connectivity structure shapes rich and lazy learning in neural circuits (ICLR 2024 · similarity 0.87)
- Scaling Laws and Tradeoffs in Recurrent Networks of Expressive Neurons (NeurIPS 2026 · similarity 0.87)
- Feedback control guides credit assignment in recurrent neural networks (NeurIPS 2024 · similarity 0.86)
- Nonlinear dynamics of localization in neural receptive fields (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.