Research map / Continual learning / Domain and prompt-based
Visual Prompt Tuning in Null Space for Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NSP², which projects visual prompt gradients into the null space of previous tasks' attention-related features (plus a prompt-distribution loss) to reduce forgetting in ViT-based class-incremental learning.
Problem
Catastrophic forgetting in prompt-tuning-based continual learning with vision transformers.
Approach
Derives consistency conditions for prompt updates in ViT layers (orthogonality to qkv-transformed image tokens and to the attention map, plus a LayerNorm distribution-invariance loss), implemented by null-space projection of prompt gradients.
Key results
- VPT-NSP² reaches 91.74% accuracy on 10-split CIFAR-100 (vs 87.27% for VPT-Seq) and 83.54% on 10-split DomainNet, with forgetting reduced to 3.28% and 8.54%.
- Surpasses the second-best method (CPrompt) by an average of 1.49% accuracy across the four benchmarks.
- On 50- and 100-task settings, VPT-NSP² outperforms baselines such as EASE and CPrompt on most benchmarks (e.g., 82.98% on 50S-CIFAR-100).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace sequential VPT fine-tuning with NSP² prompt projection | VPT-Seq · 10S-CIFAR-100 | accuracy: 87.27 -> 91.74 | Helped |
| replace sequential VPT fine-tuning with NSP² prompt projection | VPT-Seq · 20S-CIFAR-100 | accuracy: 82.36 -> 89.89 | Helped |
| replace sequential VPT fine-tuning with NSP² prompt projection | VPT-Seq · 10S-ImageNet-R | accuracy: 72.46 -> 78.88 | Helped |
| replace sequential VPT fine-tuning with NSP² prompt projection | VPT-Seq · 10S-DomainNet | accuracy: 73.28 -> 83.54 | Helped |
| add affinity projection B₁ (ablation) | VPT-NSP² · four class-incremental benchmarks | accuracy: 3.31%~9.03% improvement | Helped |
| add affinity projection B₁ (ablation) | VPT-NSP² · four class-incremental benchmarks | forgetting: 5.42%~14.76% decline | Helped |
| add aggregation projection B₂ (ablation) | VPT-NSP² · four class-incremental benchmarks | forgetting | Helped |
| add prompt-distribution loss L_LN (ablation) | VPT-NSP² · four class-incremental benchmarks | forgetting | Helped |
| add B₁, B₂, and L_LN together (ablation) | VPT-NSP² · four class-incremental benchmarks | accuracy | Helped |
Surprising findings
- The method's forgetting is not the lowest among compared methods (e.g., CODA-P has lower forgetting), which the authors attribute to a stability-plasticity trade-off.
Limitations
- The invariant prompt distribution constraint is an approximation rather than an exact solution.
- Null-space nullity must be chosen by a heuristic (maximum second derivative) and hyperparameters are tuned on a validation split.
Experimental setup
Datasets: CIFAR-100 ImageNet-R DomainNet
Benchmarks: 10-split CIFAR-100 20-split CIFAR-100 10-split ImageNet-R 10-split DomainNet
Metrics: final average accuracy final average forgetting
Baselines: L2P DualPrompt CODA-P ESN EASE CPrompt InfLoRA PGP
Builds on: VPT Adam-NSCL GPM PGP
Closest papers by meaning
- Training Consistent Mixture-of-Experts-Based Prompt Generator for Continual Learning (AAAI 2025 · similarity 0.90)
- Mixture of Experts Meets Prompt-Based Continual Learning (NeurIPS 2024 · similarity 0.90)
- Evolving Parameterized Prompt Memory for Continual Learning (AAAI 2024 · similarity 0.89)
- Convolutional Prompting meets Language Models for Continual Learning (CVPR 2024 · similarity 0.88)
- Revisiting the Power of Prompt for Visual Tuning (ICML 2024 · similarity 0.87)
- Advancing Prompt-Based Methods for Replay-Independent General Continual Learning (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.