Research map / Probabilistic and scientific modeling / Differential equations and PDEs
Deep Learning Alternatives Of The Kolmogorov Superposition Theorem
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ActNet, a KST-inspired architecture built on Laczkovich's superposition theorem (ActLayers with sinusoidal basis expansions), and evaluates it on PINN benchmarks where it beats KANs and is competitive with Siren, modified MLP and PirateNets.
Problem
Kolmogorov-Arnold Networks rest on the original KST, whose inner/outer functions are hard to implement and scale as O(d^2), and KANs have shown mixed results against MLPs; the paper seeks a more practical KST-based architecture for function approximation in PINNs.
Approach
Replace the original KST with Laczkovich's version (O(d) inner functions, valid on R^d, width m > (2+sqrt2)(2d-1)) and parametrize inner functions as linear combinations of sinusoidal basis functions, forming an ActLayer S(Λ ⊙ βB(x)) that can be read as a multi-head MLP with trainable activations or a low-rank KAN; stacking two or more ActLayers gives universality and stable activation statistics
Key results
- On Poisson with w=32, ActNet reaches relative L2 error 6.3e-2 versus 1.1e0 for KAN and 1.8e-1 for Siren.
- On advection (c=80), ActNet achieves relative L2 error 9.50e-5 versus 6.88e-4 for the JaxPi modified MLP and 5.48e-4 for PirateNet.
- On full Kuramoto-Sivashinsky, ActNet achieves 8.53e-2 versus 1.61e-1 (JaxPi) and 2.11e-1 (PirateNet), but PirateNet is better on the first time window (1.23e-5 vs 1.34e-5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ActNet instead of the strongest listed baseline | PINN · Poisson (w=16) | relative L2 error: 6.6e-2 -> 6.3e-4 | Helped |
| Use ActNet instead of the strongest listed baseline | PINN · Poisson (w=32) | relative L2 error: 1.8e-1 -> 6.3e-2 | Helped |
| Use ActNet instead of the strongest listed baseline | PINN · Helmholtz (w=16) | relative L2 error: 4.6e-2 -> 1.3e-3 | Hurt |
| Use ActNet instead of the strongest listed baseline | PINN · Helmholtz (w=32) | relative L2 error: 1.3e-1 -> 1.1e-1 | Helped |
| Use ActNet instead of Siren | PINN · Allen-Cahn | relative L2 error: 2.1e-5 -> 5.6e-5 | Hurt |
| Use ActNet instead of KAN | PINN · Poisson (w=16) | relative L2 error: 1.8e-1 -> 6.3e-4 | Helped |
| Use ActNet instead of PirateNet | PINN · Advection (c=80) | relative L2 error: 5.48e-4 -> 9.50e-5 | Helped |
| Use ActNet instead of PirateNet | PINN · Kuramoto–Sivashinsky (first time window) | relative L2 error: 1.23e-5 -> 1.34e-5 | Hurt |
| Use ActNet instead of PirateNet | PINN · Kuramoto–Sivashinsky (full solution) | relative L2 error: 2.11e-1 -> 8.53e-2 | Helped |
| Allow unconstrained lambda values and non-monotonic inner functions rather than enforcing positive lambda and increasing inner functions (ablation) | ActNet · PINN experiments | performance | No clear effect |
Surprising findings
- Siren performs better than ActNet on the Allen-Cahn benchmark at larger network sizes and in best relative L2 error (2.1e-5 vs 5.6e-5).
Limitations
- ActNet is slower than plain MLPs (about 2-3x), though faster than KANs.
- Evaluated only on PINN (data-free) settings; data-driven performance and use inside larger architectures are untested.
- Sensitivity of trainable basis frequencies and phases requires adaptive gradient clipping.
Experimental setup
Datasets: Poisson PDE (synthetic forcing terms) Helmholtz PDE (synthetic) Allen-Cahn PDE Advection PDE Kuramoto-Sivashinsky PDE
Benchmarks: Poisson equation Helmholtz equation Allen-Cahn equation Advection equation (c=80) Kuramoto-Sivashinsky equation
Metrics: relative L2 error residual loss
Baselines: KAN Siren modified MLP (JaxPi) PirateNet MLP
Builds on: Kolmogorov-Arnold Networks Laczkovich superposition theorem PINNs Siren
Closest papers by meaning
- KAN: Kolmogorov–Arnold Networks (ICLR 2025 · similarity 0.87)
- Incorporating Arbitrary Matrix Group Equivariance into KANs (ICML 2025 · similarity 0.86)
- PowerMLP: An Efficient Version of KAN (AAAI 2025 · similarity 0.86)
- On the expressiveness and spectral bias of KANs (ICLR 2025 · similarity 0.85)
- Deep-Koopman-KANDy: Dictionary Discovery for Deep-Koopman Operators with Kolmogorov-Arnold Networks for Dynamics (NeurIPS 2026 · similarity 0.85)
- Generalization Bounds and Model Complexity for Kolmogorov–Arnold Networks (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.