Research map / Continual learning / Online and replay-based
Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that Kolmogorov-Arnold Networks still catastrophically forget on more complex task-incremental regression benchmarks, and proposes WiseKAN, a parameter-isolation method with hierarchical pruning tailored to KAN, which together with EWC/SI makes KAN competitive with or better than MLP.
Problem
Whether KAN avoid catastrophic forgetting in continual learning beyond the simple 1D experiment of the original KAN paper, especially on task-incremental regression with overlapping inputs between tasks.
Approach
Explains KAN forgetting via "spline input overlap" and weight-scale parameters, then adapts parameter isolation (WiseKAN) with a visibility rule, freezing rule, and hierarchical spline/coefficient pruning with early stopping; also combines KAN with EWC and SI regularisation.
Key results
- On TI Feynman, WiseKAN reaches R2 0.9994 with zero backward transfer versus 0.7046 for unmodified KAN (BWT -0.213) and 0.7893 for MLP.
- On TI Europe Wind Farm, unmodified KAN forgets more than MLP (BWT -0.134 vs. -0.067), while EWC-KAN (0.709) outperforms EWC-MLP (0.680) on average R2.
- On TI River Radar, unmodified KAN has R2 0.159 with BWT -0.283 versus MLP at 0.109 with BWT -0.400; PackNet reaches 0.587 and WiseKAN 0.539.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add WiseKAN parameter isolation to KAN | KAN · TI Feynman | R2: 0.7046 ± .0158 -> 0.9994 ± .0002 | Helped |
| Add WiseKAN parameter isolation to KAN | KAN · TI Europe Wind Farm | R2: 0.629 ± .012 -> 0.794 ± .006 | Helped |
| Add WiseKAN parameter isolation to KAN | KAN · TI River Radar | R2: 0.159 ± .019 -> 0.539 ± .059 | Helped |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI Feynman | R2: 0.7893 ± .0122 -> 0.7046 ± .0158 | Hurt |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI Europe Wind Farm | R2: 0.651 ± .014 -> 0.629 ± .012 | Hurt |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI River Radar | R2: 0.109 ± .062 -> 0.159 ± .019 | Helped |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI Feynman | R2bwt: −0.120 ± .013 -> −0.213 ± .017 | Hurt |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI Europe Wind Farm | R2bwt: −0.067 ± .017 -> -0.134 ± .019 | Hurt |
| Use unmodified KAN instead of unmodified MLP | continual learning model · TI River Radar | R2bwt: −0.400 ± .085 -> −0.283 ± .024 | Helped |
| Use SI with KAN instead of SI with MLP | SI continual learner · TI Feynman | R2: 0.9896 ± .0012 -> 0.9992 ± .0002 | Helped |
| Use SI with KAN instead of SI with MLP | SI continual learner · TI Europe Wind Farm | R2: 0.699 ± .002 -> 0.733 ± .002 | Helped |
| Use SI with KAN instead of SI with MLP | SI continual learner · TI River Radar | R2: 0.446 ± .012 -> 0.464 ± .009 | Helped |
Surprising findings
- Unmodified KAN forget more than MLP on the tested regression benchmarks, contrary to the claim that KAN avoid catastrophic forgetting.
- KAN variants of EWC and SI often outperform their MLP counterparts, mainly at higher parameter counts.
Limitations
- Restricted to task-incremental setting; task-ID inference left for future work.
- WiseKAN stops learning when finite capacity is exhausted, favouring stability over plasticity.
- Only classic regularisation and a parameter isolation baseline compared; replay, representation and optimisation methods excluded.
Experimental setup
Datasets: Feynman Symbolic Regression Database Europe Wind Farm River Radar
Benchmarks: TI Feynman TI Europe Wind Farm TI River Radar
Metrics: R2 average R2 backward transfer R2 plasticity
Baselines: MLP KAN EWC SI PackNet WiseMLP Joint-MLP Joint-KAN
Builds on: Kolmogorov-Arnold Networks PackNet Elastic weight consolidation Synaptic intelligence Zhu and Gupta gradual pruning
Closest papers by meaning
- KAC: Kolmogorov-Arnold Classifier for Continual Learning (CVPR 2025 · similarity 0.89)
- KAN: Kolmogorov–Arnold Networks (ICLR 2025 · similarity 0.88)
- The Joint Effect of Task Similarity and Overparameterization on Catastrophic Forgetting — An Analytical Model (ICLR 2024 · similarity 0.88)
- Do Your Best and Get Enough Rest for Continual Learning (CVPR 2025 · similarity 0.88)
- Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning (ICLR 2025 · similarity 0.88)
- Towards Understanding Catastrophic Forgetting in Two-layer Convolutional Neural Networks (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.