Research map / Continual learning / Domain and prompt-based
Consistent Prompting for Rehearsal-Free Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Consistent Prompting (CPrompt), which aligns prompt-based rehearsal-free class-incremental learning training with testing via classifier consistency learning and prompt consistency learning with multi-key prompt selection.
Problem
Prompt-based rehearsal-free continual learning suffers from training-testing inconsistency: classifiers are trained only for the current task while inference uses all classifiers, and the prompt selected at test time may not match the training prompt.
Approach
Expose the current task prompt to all existing classifiers with a smooth entropy-based regularization (CCL), train the current classifier under randomly selected prompts with an auxiliary classifier (PCL), and map each class to its own key for more accurate prompt selection (multi-key).
Key results
- On Split StanfordCars 10-task, CPrompt reaches 66.77% Last-acc vs 62.24% for CODAPrompt.
- On Split ImageNet-R 20-task, CPrompt reaches 74.79% Last-acc vs 72.25% for CODAPrompt.
- On Split CIFAR-100 10-task, CPrompt reaches 87.82% Last-acc and 92.53% Avg-acc, outperforming all prompt-based competitors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CPrompt instead of CODAprompt | ViT-B/16 continual learning · Split StanfordCars 10-task | Last-acc: 62.24±0.14 -> 66.77±0.37 | Helped |
| Use CPrompt instead of CODAprompt | ViT-B/16 continual learning · Split StanfordCars 20-task | Avg-acc: 63.78±1.46 -> 68.74±0.83 | Helped |
| Use CPrompt instead of CODAprompt | ViT-B/16 continual learning · Split ImageNet-R 20-task | Last-acc: 72.25±0.78 -> 74.79±0.28 | Helped |
| Use CPrompt instead of DualPrompt | ViT-B/16 continual learning · Split CIFAR-100 10-task | Avg-acc: 90.82±1.47 -> 92.53±0.23 | Helped |
| Use CPrompt instead of DualPrompt | ViT-B/16 continual learning · Split DomainNet 10-task | Last-acc: 81.70±0.78 -> 82.97±0.34 | Helped |
| Add CCL (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 61.96 -> 62.52 | Helped |
| Add CCL (ablation) | CPrompt · Split StanfordCars 10-task | Avg-acc: 73.10 -> 73.42 | Helped |
| Add PCL without multi-key (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 61.96 -> 65.18 | Helped |
| Add the multi-key mechanism (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 61.96 -> 63.66 | Helped |
| Add CCL and PCL without multi-key (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 62.52 -> 66.37 | Helped |
| Remove the auxiliary loss Laux (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 66.77 -> 60.34 | Hurt |
| Replace auxiliary classifier Caux with Ct (ablation) | CPrompt · Split StanfordCars 10-task | Last-acc: 66.77 -> 64.17 | Hurt |
Surprising findings
- Multi-key raises prompt selection accuracy by over 20% (19.44% to 39.83%) but prompt selection remains below 40% on StanfordCars.
Limitations
- Prompt selection accuracy remains low (under 40%) on StanfordCars, limiting performance.
- Gains are evaluated only against prompt-based methods on ViT-B/16 with ImageNet pre-training.
Experimental setup
Datasets: StanfordCars ImageNet-R DomainNet CIFAR-100
Benchmarks: Split StanfordCars Split ImageNet-R Split DomainNet Split CIFAR-100
Metrics: Last-acc Avg-acc forgetting
Baselines: L2P DualPrompt ESN CODAPrompt
Builds on: L2P DualPrompt CODA-Prompt ViT
Closest papers by meaning
- Training Consistent Mixture-of-Experts-Based Prompt Generator for Continual Learning (AAAI 2025 · similarity 0.92)
- REP: Resource-Efficient Prompting for Rehearsal-Free Continual Learning (NeurIPS 2025 · similarity 0.90)
- Evolving Parameterized Prompt Memory for Continual Learning (AAAI 2024 · similarity 0.89)
- Convolutional Prompting meets Language Models for Continual Learning (CVPR 2024 · similarity 0.89)
- Vector Quantization Prompting for Continual Learning (NeurIPS 2024 · similarity 0.89)
- Mixture of Experts Meets Prompt-Based Continual Learning (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.