Research map / Continual learning / Class-incremental
PEARL: Input-Agnostic Prompt Enhancement with Negative Feedback Regulation for Class-Incremental Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PEARL, an input-agnostic global prompt encoder with negative-feedback adaptive momentum updates for pre-trained-model-based class-incremental learning, reporting state-of-the-art results across six benchmarks.
Problem
Prompt-based class-incremental learning with pre-trained ViTs relies on input-dependent query-select prompt pools, which mix task knowledge inconsistently and cause catastrophic forgetting.
Approach
Replaces query-select prompt pools with a session-sharing prompt encoder that produces one input-agnostic prompt, plus a momentum update whose weight is adapted by a negative-feedback signal (MAE between current and previous logits on old classes) instead of a fixed weight.
Key results
- PEARL achieves average accuracy 80.26 across six benchmarks with ViT-B/16-IN21K, versus 78.83 for second-best HiDe-Prompt (average final accuracy 78.83 vs PEARL's 2.24 point average gain over RanPAC on average accuracy as stated).
- On CUB 20-tasks, PEARL reaches 94.48 average accuracy vs 93.13 for RanPAC.
- Adding the prompt encoder alone to L2P, DualPrompt and CODA-Prompt degrades performance strongly; combining it with fixed-weight momentum recovers and surpasses baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PEARL instead of the strongest baseline | CIL methods with ViT-B/16-IN21K · VTAB 5-tasks | average accuracy: 93.61 -> 96.52 | Helped |
| Use PEARL instead of the strongest baseline | CIL methods with ViT-B/16-IN21K · CIFAR 20-tasks | final accuracy: 89.92 -> 89.02 | Hurt |
| Add a prompt encoder without momentum update (ablation) | L2P · ImageNet-A 10-tasks | average accuracy: 53.36 -> 3.00 | Hurt |
| Add a prompt encoder without momentum update (ablation) | L2P · VTAB 5-tasks | average accuracy: 80.84 -> 5.36 | Hurt |
| Add fixed-weight momentum update to the prompt encoder (ablation) | L2P · ImageNet-A 10-tasks | average accuracy: 3.00 -> 58.16 | Helped |
| Add fixed-weight momentum update to the prompt encoder (ablation) | L2P · VTAB 5-tasks | average accuracy: 5.36 -> 88.51 | Helped |
| Replace continual positional encoding with segmented positional encoding (ablation) | PEARL · ImageNet-A 10-tasks | average accuracy: 66.91 -> 67.65 | Helped |
| Replace continual positional encoding with segmented positional encoding (ablation) | PEARL · VTAB 5-tasks | average accuracy: 96.47 -> 96.59 | Helped |
| Adapt momentum weight α with NKA instead of fixing it (ablation) | PEARL · CUB-200 10-tasks | average accuracy: 60.04 -> 81.56 | Helped |
| Adapt momentum weight α with NKA instead of fixing it (ablation) | PEARL · CUB-200 10-tasks | final accuracy: 70.31 -> 84.73 | Helped |
Surprising findings
- Using the prompt encoder alone (without momentum update) causes severe performance drops for L2P and CODA-Prompt, e.g., CODA-Prompt on IN-A falls to 15.93 average accuracy.
Limitations
- Relies on a fixed set of hyperparameters (alpha0, lambda, sigmoid bounds) whose sensitivity is only partly studied.
- Fixed-weight momentum update fails at low initial alpha, indicating sensitivity that NKA only partly alleviates.
Experimental setup
Datasets: CIFAR-100 CUB-200 ImageNet-R ImageNet-A Omnibenchmark VTAB
Benchmarks: CIFAR-100 (20 tasks) CUB-200 (20 tasks) ImageNet-R (40 tasks) ImageNet-A (10 tasks) Omnibenchmark (10 tasks) VTAB (5 tasks)
Metrics: average accuracy final accuracy
Baselines: L2P DualPrompt CODA-Prompt SimpleCIL ADAM RanPAC EASE HiDe-Prompt ESN
Builds on: CODA-Prompt L2P DualPrompt RanPAC prefix tuning
Closest papers by meaning
- Knowledge Memorization and Rumination for Pre-trained Model-based Class-Incremental Learning (CVPR 2025 · similarity 0.89)
- SEC-Prompt:SEmantic Complementary Prompting for Few-Shot Class-Incremental Learning (CVPR 2025 · similarity 0.89)
- CAPrompt: Cyclic Prompt Aggregation for Pre-Trained Model Based Class Incremental Learning (AAAI 2025 · similarity 0.88)
- Make Domain Shift a Catastrophic Forgetting Alleviator in Class-Incremental Learning (AAAI 2025 · similarity 0.87)
- Long-Tail Class Incremental Learning via Independent Sub-prototype Construction (CVPR 2024 · similarity 0.87)
- Compress to One Point: Neural Collapse for Pre-Trained Model-Based Class-Incremental Learning (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.