Research map / Continual learning / Online and replay-based
KAC: Kolmogorov-Arnold Classifier for Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes the Kolmogorov-Arnold Classifier (KAC), a drop-in replacement for linear classifiers in prompt-based class-incremental learning that uses KAN layers with Gaussian RBF basis functions to reduce forgetting.
Problem
Linear classifiers in continual learning struggle to keep a stable classification space as new tasks arrive; naive KAN layers with B-spline bases perform worse than linear classifiers in class-incremental learning.
Approach
Replace the linear classifier with a KAN-style layer whose univariate activations are fixed Gaussian RBFs (layer norm, N=4 RBF centers per channel, class-specific weights), so each channel's class-specific activation range is localized and new-task updates touch only relevant parts of the feature space.
Key results
- On ImageNet-R 20 steps, CODAPrompt with KAC improves average accuracy from 80.92 to 83.59 and last accuracy from 74.40 to 77.94.
- On CUB200 20 steps, L2P with KAC improves last accuracy from 51.78 to 70.13 (+18.35).
- On DomainNet domain-incremental learning, KAC improves average accuracy by about 1 point across the four prompt-based methods (e.g., L2P 57.78 to 59.79).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the linear classifier with KAC | CPrompt · ImageNet-R, 40 steps | Last: 70.07 -> 72.05 | Helped |
| replace the linear classifier with KAC | CPrompt · CUB200, 40 steps | Last: 64.80 -> 76.51 | Helped |
| replace the linear classifier with KAC | CODAPrompt · DomainNet, 6 domains | Avg: 61.61 -> 62.78 | Helped |
| replace KAC spline functions with a trainable MLP (ablation) | CODAPrompt w/ KAC · ImageNet-R, 20 steps | Avg: 83.59 -> 80.56 | Hurt |
| replace KAC spline functions with a trainable MLP (ablation) | CODAPrompt w/ KAC · ImageNet-R, 20 steps | Last: 77.94 -> 73.59 | Hurt |
| replace KAC spline functions with a fixed random MLP (ablation) | CODAPrompt w/ KAC · ImageNet-R, 20 steps | Avg: 83.59 -> 65.87 | Hurt |
| replace KAC spline functions with a fixed random MLP (ablation) | CODAPrompt w/ KAC · ImageNet-R, 20 steps | Last: 77.94 -> 51.03 | Hurt |
| increase the number of basis functions (ablation) | KAC · ImageNet-R, 20 steps | average incremental accuracy | No clear effect |
Surprising findings
- A naive KAN layer with B-spline bases reduces performance when used as a continual classifier, even though KAN is reported to resist forgetting in simple regression.
- Replacing RBFs with a parameter-matched MLP (trained or fixed) does not help, indicating gains come from the KAN structure rather than extra capacity.
Limitations
- Gains are shown only for prompt-based CIL methods with a ViT-B/16 backbone; generalization to other architectures is not demonstrated.
- Experiments are in a non-exemplar setting; behavior with replay is not reported in the main text.
- Gains vary across methods (e.g., CPrompt on ImageNet-R improves by under 1 point at some steps).
Experimental setup
Datasets: ImageNet-R CUB200 DomainNet
Benchmarks: ImageNet-R CUB200 DomainNet
Metrics: average incremental accuracy last accuracy
Baselines: L2P DualPrompt CODAPrompt CPrompt MLP classifier (fixed and trained) KAN with B-spline
Builds on: Kolmogorov-Arnold Networks FastKAN L2P DualPrompt CODA-Prompt CPrompt
Closest papers by meaning
- Kolmogorov-Arnold Networks Still Catastrophically Forget but Differently from MLP (AAAI 2025 · similarity 0.89)
- Improving Continual Learning Performance and Efficiency with Auxiliary Classifiers (ICML 2025 · similarity 0.89)
- Continual Learning Using a Kernel-Based Method Over Foundation Models (AAAI 2025 · similarity 0.88)
- Continual learning with the neural tangent ensemble (NeurIPS 2024 · similarity 0.88)
- Learning to Continually Learn with the Bayesian Principle (ICML 2024 · similarity 0.87)
- STAR: Stability-Inducing Weight Perturbation for Continual Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.