Research map / Learning theory and algorithms / Prediction and decision losses
Decomposing stimulus-specific sensory neural information via diffusion models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an axiomatically motivated, local stimulus- and feature-wise decomposition of neural mutual information, estimated efficiently with diffusion models via an exact Shannon-Fisher integral identity.
Problem
Decomposing mutual information between stimuli and neural responses into per-stimulus and per-feature contributions is ill-posed, and existing decompositions violate locality, positivity, or additivity and are intractable for high-dimensional stimuli.
Approach
Derives an exact identity I(R;X) = 1/2 Trace ∫_0^∞ E[J(X_γ)] dγ linking mutual information to Fisher information of Gaussian-noise-corrupted stimuli, giving a stimulus-wise decomposition I_local(x) that satisfies completeness, locality, positivity, additivity, and (for features) insensitivity to irrelevant features; it is estimated with conditional and unconditional diffusion models.
Key results
- On a 1-D toy model, I_local differs from baseline attributions only where the tuning curves differ (locality), whereas I_surp, I_SSI, I_sp show global changes.
- Under a bimodal prior, I_local peaks near x=0 where posterior is most sensitive and is flat under a Gaussian prior, while several baselines fail this check.
- For the pointwise data processing inequality, I_local decreased at every x after a non-invertible transform |r|, while I_SSI and I_sp increased at some x and I_surp respected it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed local information decomposition instead of global stimulus-specific attribution methods | model visual neurons · one-dimensional Gaussian-prior stimulus | locality of stimulus attribution | Helped |
| Use I_local(x) to attribute information under a bimodal stimulus prior | linear-Gaussian neuron · bimodal Gaussian-mixture prior | alignment with posterior sensitivity | Helped |
| Apply the non-invertible transformation r' = |r| | linear-Gaussian neuron · bimodal prior | pointwise data-processing inequality | Helped |
| Use diffusion-based I_local instead of diagonal Fisher information | 49-neuron visual model · MNIST | pixel-wise information localization | Mixed |
| Use the proposed local information decomposition | model of retinal ganglion cells · Tiny ImageNet natural images | pixel-wise information localization | Helped |
| Average the first Kong diffusion decomposition over neural responses | Kong et al. decomposition · stimulus-wise attribution | locality | Hurt |
| Average the second Kong diffusion decomposition over neural responses | Kong et al. decomposition · multiple-neuron measurements | pointwise additivity | Hurt |
Surprising findings
- Averaging the Kong et al. decompositions over p(R|x) yields a non-local measure (equal to specific surprise) in one case and a non-additive measure in the other.
- The Brunel-Nadal style Fisher-based approximation is noted to become inaccurate at high noise or when stimulus dimensions exceed neuron count, where it returns minus infinity.
Limitations
- Uniqueness of the decomposition under the axioms is not proven.
- Experiments use simple simulated LNP neural models rather than fitted biological data, though a retina model is used for natural images.
- Monte Carlo approximation replaces sampling r from p(R|x_gamma) with samples from p(R|x̂), an approximation adopted from Chung et al.
Experimental setup
Datasets: MNIST Tiny ImageNet simulated LNP neural population (49 neurons) retinal ganglion cell recordings (41 RGCs) for retina model
Benchmarks: MNIST Tiny ImageNet
Metrics: pointwise information attribution I_local(x) Fisher information comparison axiom satisfaction (completeness, locality, positivity, addi
Baselines: Fisher information stimulus-specific information (I_SSI) specific information (I_sp) specific surprise (I_surp) coordinate-invariant SSI (I_CiSSI) Kong et al. decompositions
Builds on: Fisher information Tweedie's formula DDPM integrated gradients Brunel & Nadal low-noise approximation
Closest papers by meaning
- Information Geometry of the Retinal Representation Manifold (NeurIPS 2023 · similarity 0.90)
- An Information-Theoretical Framework For Optimizing Experimental Design To Distinguish Probabilistic Neural Codes (ICLR 2026 · similarity 0.89)
- Neurons as Detectors of Coherent Sets in Sensory Dynamics (NeurIPS 2025 · similarity 0.86)
- Flexible Context-Driven Sensory Processing in Dynamical Vision Models (NeurIPS 2024 · similarity 0.86)
- Complex priors and flexible inference in recurrent circuits with dendritic nonlinearities (ICLR 2024 · similarity 0.86)
- Modeling Dynamic Neural Activity by combining Naturalistic Video Stimuli and Stimulus-independent Latent Factors (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.