Research map / Architectures and efficiency / RNNs and normalization
KAN: Kolmogorov–Arnold Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Kolmogorov-Arnold Networks (KANs), which place learnable spline-parametrized univariate activations on edges instead of fixed activations on nodes, and shows better accuracy, scaling and interpretability than MLPs on small-scale function fitting, PDE and science tasks.
Problem
MLPs use fixed node activations and linear weight matrices, which are parameter-hungry and hard to interpret; the paper asks whether a network built from the Kolmogorov-Arnold representation theorem can be more accurate and interpretable for small-scale AI+Science tasks.
Approach
Replace every linear weight with a learnable univariate B-spline function on edges (plus a SiLU residual branch) and sum at nodes, generalizing the depth-2 Kolmogorov-Arnold representation to arbitrary widths and depths, with grid extension, sparsification/pruning and symbolification for interpretability.
Key results
- KANs reach test RMSE scaling near the predicted alpha=4 on toy examples while MLPs plateau; a 2-layer width-10 KAN is about 100x more accurate than a 4-layer width-100 MLP on the Poisson PDE (1e-7 vs 1e-5 MSE) with 100x fewer parameters (1e2 vs 1e4).
- On 15 special functions, KANs achieve lower test RMSE than MLPs at equal parameter counts; e.g., Bessel Y_v best KAN RMSE 1.49e-5 vs MLP 3.45e-4.
- On the knot-theory signature task, an extremely small [17,1,14] KAN reaches 81.6% test accuracy, rediscovering the dependence on meridinal distance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a KAN instead of an MLP | Function fitting · Jacobian elliptic functions | test RMSE: 6.48\times 10^{-4} -> 1.33\times 10^{-4} | Helped |
| Use a KAN instead of an MLP | Function fitting · Associated Legendre function (m=0) | test RMSE: 1.74\times 10^{-2} -> 5.25\times 10^{-5} | Helped |
| Use a KAN instead of an MLP | Function fitting · Bessel function of the second kind | test RMSE: 3.45\times 10^{-4} -> 1.49\times 10^{-5} | Helped |
| Use a KAN instead of an MLP | Function fitting · spherical harmonics (m=1,n=2) | test RMSE: 1.84\times 10^{-3} -> 1.50\times 10^{-5} | Helped |
| Use a pruned KAN instead of an MLP | Feynman equation fitting · Feynman I.6.2 | test RMSE: 1.45\times 10^{-4} -> 2.86\times 10^{-5} | Helped |
| Prune the KAN (ablation) | Feynman I.9.18 KAN · Feynman I.9.18 | test RMSE: 1.48\times 10^{-3} -> 8.62\times 10^{-3} | Hurt |
| Use a 2-layer width-10 KAN instead of a 4-layer width-100 MLP | Poisson equation PINN · Poisson equation | MSE: 10^{-5} -> 10^{-7} | Helped |
| Fine-grain the spline grids using grid extension | KAN function fitting · Toy function fitting | training loss | Helped |
| Increase grid size to 1000 points (ablation) | KAN function fitting · Toy function fitting | training loss | Hurt |
| Use a smaller [2,1,1] KAN instead of a [2,5,1] KAN (ablation) | Toy function fitting · f(x,y)=exp(sin(πx)+y²) | test loss | Helped |
| Use a deeper 3-layer KAN instead of a 2-layer KAN (ablation) | KAN function fitting · Four-dimensional toy example | test RMSE | Helped |
| Add learnable spline activations to the SIREN network | SIREN/LAN image representation · Image fitting | PSNR | Helped |
Surprising findings
- Auto-discovered KANs for relativistic velocity addition are only 2 layers deep, versus 5 in the human construction, matching the rapidity trick.
- On Feynman equations MLPs and KANs perform comparably on average, so KAN gains are concentrated on special functions and PDEs.
Limitations
- KANs are roughly 10x slower to train than MLPs in the authors' implementation, and the experiments are small-scale.
- Scaling results rely on LBFGS vanilla setups; comparison in advanced training setups is left for future work.
- Interpretability and accuracy advantages are demonstrated on toy and small scientific tasks, not large-scale deep learning.
Experimental setup
Baselines: MLP SIREN Sharma & Kaplan scaling theory
Builds on: Kolmogorov-Arnold representation theorem MLP physics-informed neural networks SIREN
Closest papers by meaning
- On the expressiveness and spectral bias of KANs (ICLR 2025 · similarity 0.95)
- PowerMLP: An Efficient Version of KAN (AAAI 2025 · similarity 0.91)
- Kolmogorov-Arnold Transformer (ICLR 2025 · similarity 0.90)
- Generalization Bounds and Model Complexity for Kolmogorov–Arnold Networks (ICLR 2025 · similarity 0.90)
- Incorporating Arbitrary Matrix Group Equivariance into KANs (ICML 2025 · similarity 0.89)
- Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.