Research map / Architectures and efficiency / RNNs and normalization
Identifiability of Deep Polynomial Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proves finite identifiability of deep polynomial neural networks (with and without biases) by reducing deep identifiability to 2-layer blocks via a localization theorem linked to partially symmetric tensor decompositions and Kruskal-type uniqueness.
Problem
Identifiability of deep polynomial neural networks (PNNs), i.e., whether parameters can be recovered from the network function up to neuron permutations and rescalings, which is tied to the dimension of neurovarieties and activation thresholds.
Approach
A localization theorem: an L-layer monomial-activation PNN is finitely identifiable if every consecutive 2-layer block is finitely identifiable for some subset of inputs; 2-layer identifiability is derived from Kruskal-type uniqueness of partially symmetric CPDs, and biases are handled by homogenization.
Key results
- Pyramidal (non-increasing width) polynomial networks are finitely identifiable for degrees at least 2 (or at least 3 in the last layer for single output).
- Activation thresholds for identifiability are bounded linearly by r_l >= 2 d_l - 1, improving the previous quadratic bound.
- Settles a conjecture that neurovarieties of PNNs with non-increasing widths and quadratic-or-higher degrees have expected dimension; bottleneck encoder-decoder networks are identifiable under a decoder width/degree condition.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Require every consecutive two-layer block to be finitely identifiable on the specified subset of inputs | Deep hPNNs · General deep architectures | finite identifiability | Helped |
| Use non-increasing hidden-layer widths with activation degrees at least 2 (at least 3 in the last layer for a single output) | hPNNs · Pyramidal architectures | finite identifiability | Helped |
| Set each activation degree to at least 2d_l - 1 | hPNNs · Fixed layer widths | finite identifiability | Helped |
| Keep decoder widths within the bottleneck-width condition d_l/r_l <= d_b - 1 | Bottleneck hPNNs · Encoder-decoder architectures | finite identifiability | Helped |
| Homogenize the network and apply the stated identifiability conditions to the extended widths | PNNs with biases · General deep architectures | finite identifiability | Helped |
| Increase any hidden-layer width by one neuron | hPNNs · Any architecture | finite identifiability | Hurt |
| Use degree 2 for pyramidal PNNs with biases | PNNs with biases · Pyramidal architectures | finite identifiability | Helped |
Surprising findings
- A deep PNN can be non-unique even when each of its 2-layer subnetworks is unique, so global identifiability conjectures do not transfer to uniqueness.
Limitations
- Results only establish finite identifiability for deep PNNs, not global identifiability (left as conjecture).
- Restricted to monomial activation functions.
- Bounds from basic Kruskal conditions are not the sharpest possible.
Experimental setup
Builds on: Kruskal uniqueness theorem neurovariety framework for polynomial networks partially symmetric CPD
Closest papers by meaning
- On the Expressive Power of Deep Polynomial Neural Networks (NeurIPS 2019 · similarity 0.91)
- Verifying Properties of Binary Neural Networks Using Sparse Polynomial Optimization (ICLR 2025 · similarity 0.86)
- Exponent Relaxation of Polynomial Zonotopes and Its Applications in Formal Neural Network Verification (AAAI 2024 · similarity 0.85)
- Multilinear Operator Networks (ICLR 2024 · similarity 0.85)
- How Universal Polynomial Bases Enhance Spectral Graph Neural Networks: Heterophily, Over-smoothing, and Over-squashing (ICML 2024 · similarity 0.85)
- Beyond ICA: Identifiability by Symmetry Breaking (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.