Research map / Continual learning / Class-incremental
F-OAL: Forward-only Online Analytic Learning with Fast Training and Low Memory Footprint in Class Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes F-OAL, an exemplar-free, forward-only online class-incremental learner that freezes a pre-trained ViT with feature fusion and random sigmoid expansion, and updates a linear analytic classifier via recursive least squares.
Problem
Online class incremental learning (OCIL) requires learning from mini-batch streams in one pass without storing exemplars, while avoiding catastrophic forgetting, recency bias, and high time/memory cost of back-propagation.
Approach
Freeze a pre-trained ViT with averaged block-output feature fusion and a random projection plus sigmoid (Smooth Projection), then solve the linear classifier in closed form and update it batch-wise with recursive least squares (Woodbury identity), so no back-propagation or exemplars are needed and the result matches joint training.
Key results
- F-OAL reaches 91.1% average accuracy on CIFAR-100 and 62.2% on FGVCAircraft, the best among exemplar-free methods on most datasets.
- On Tiny-ImageNet F-OAL (91.2%) trails exemplar-free EASE (92.0%) in average accuracy by 0.8 points.
- Training time on DTD is 8 seconds for F-OAL versus 40 to 278 seconds for most baselines (Table of training times).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · FGVCAircraft | A_avg (%): 55.6 -> 62.2 | Helped |
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · DTD | A_avg (%): 76.0 -> 82.8 | Helped |
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · Country211 | A_avg (%): 17.8 -> 24.4 | Helped |
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · CIFAR-100 | A_avg (%): 92.4 -> 91.1 | Hurt |
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · CORe50 | A_avg (%): 97.1 -> 96.3 | Hurt |
| use F-OAL instead of the strongest listed baseline | OCIL with ViT-B · Tiny-ImageNet | A_avg (%): 92.0 -> 91.2 | Hurt |
| add Feature Fusion (ablation) | F-OAL · CIFAR-100 | A_avg (%): 90.6 -> 91.1 | Helped |
| add Feature Fusion (ablation) | F-OAL · CORe50 | A_avg (%): 95.3 -> 96.3 | Helped |
| add Feature Fusion (ablation) | F-OAL · FGVCAircraft | A_avg (%): 60.9 -> 62.2 | Helped |
| add Feature Fusion (ablation) | F-OAL · DTD | A_avg (%): 80.5 -> 82.8 | Helped |
| add Feature Fusion (ablation) | F-OAL · Country211 | A_avg (%): 21.3 -> 24.4 | Helped |
| add Smooth Projection (ablation) | F-OAL · FGVCAircraft | A_avg (%): 58.7 -> 62.2 | Helped |
Surprising findings
- Replacing the analytic classifier with a back-propagation-trained classifier of the same structure drops CIFAR-100 accuracy from 91.1% to 32.4%.
Limitations
- Relies heavily on a powerful pre-trained backbone; the authors note that it is not tested on long-tail distributions.
- Projection size is limited by cubic matrix inversion cost; 2,000-dimensional projection is roughly twice as slow.
- Results depend on regularization term gamma, which degrades sharply on fine-grained datasets when gamma is small (e.g. FGVCAircraft drops to 22.4% at gamma=1e-3).
Experimental setup
Datasets: CIFAR-100 CORe50 FGVCAircraft DTD Tiny-ImageNet Country211 ImageNet-1K
Benchmarks: CIFAR-100 CORe50 FGVCAircraft DTD Tiny-ImageNet Country211
Metrics: average accuracy last task accuracy forgetting rate training time peak GPU memory
Baselines: iCaRL ER ASER SCR DVC PCR LwF EWC EASE LAE SLCA
Builds on: Analytic Class Incremental Learning (ACIL) GKEAL Moore-Penrose block-wise recursive algorithm
Closest papers by meaning
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.87)
- Salient Frequency-aware Exemplar Compression for Resource-constrained Online Continual Learning (AAAI 2025 · similarity 0.87)
- GACL: Exemplar-Free Generalized Analytic Continual Learning (NeurIPS 2024 · similarity 0.87)
- OVOR: OnePrompt with Virtual Outlier Regularization for Rehearsal-Free Class-Incremental Learning (ICLR 2024 · similarity 0.86)
- DS-AL: A Dual-Stream Analytic Learning for Exemplar-Free Class-Incremental Learning (AAAI 2024 · similarity 0.86)
- Non-exemplar Online Class-Incremental Continual Learning via Dual-Prototype Self-Augment and Refinement (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.