Research map / Probabilistic and scientific modeling / Molecules and proteins
Where We Have Arrived in Proving the Emergence of Sparse Interaction Primitives in DNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves three conditions under which sparse interaction primitives (symbolic concepts) emerge in well-trained DNNs, based on derivative, occlusion, and related properties of the network.
Problem
Explaining why well-trained deep neural networks encode sparse, symbolic inference patterns (interaction primitives); a formal proof of when such emergence happens.
Approach
Derives three sufficient conditions for emergence of sparse interaction primitives: vanishing high-order input derivatives, robustness to occlusion of inputs (usable on occluded samples), and a further condition stated in the truncated abstract.
Key results
- Proves that high-order derivatives of the network output with respect to input variables are all zero is a condition for emergence of sparse interaction primitives.
Closest papers by meaning
- Defining and extracting generalizable interaction primitives from DNNs (ICLR 2024 · similarity 0.92)
- Towards the Dynamics of a DNN Learning Symbolic Interactions (NeurIPS 2024 · similarity 0.89)
- Monitoring Primitive Interactions During the Training of DNNs (AAAI 2025 · similarity 0.89)
- Layerwise Change of Knowledge in Neural Networks (ICML 2024 · similarity 0.88)
- Position: A Theory of Deep Learning Must Include Compositional Sparsity (ICML 2025 · similarity 0.86)
- Random Sparse Lifts: Construction, Analysis and Convergence of finite sparse networks (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.