Research map / Continual learning / Online and replay-based
InTAct: Interval-based Task Activation Consolidation for Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes InTAct, which protects prior-task activation hypercubes (per-neuron quantile intervals on CLS tokens and classifier features) with interval-arithmetic drift losses, integrated into CODA-Prompt for domain-incremental learning.
Problem
Prompt-based continual learning remains vulnerable to representation drift in shared components (classifier head and prompt keys/prompts) under domain-incremental shifts, causing forgetting without replay.
Approach
Instead of constraining weights, build per-neuron activation intervals (quantile-based hypercubes) for prior-task CLS tokens and classifier features, and penalize worst-case output drift over these hypercubes via interval arithmetic; add a region-gated feature distillation for prompts and compactness/alignment regularizers to keep hypercubes tight.
Key results
- InTAct integrated into CODA-Prompt reaches 72.74% average AA across four DIL benchmarks, the highest in the comparison table (KA-Prompt 71.30%).
- On ImageNet-R InTAct gets 74.23% AA, about 7.7 p.p. above KA-Prompt (66.51%).
- Ablation on ImageNet-R: removing the IntDrift loss drops AA by 3.48 p.p. to 70.75%.
Surprising findings
- KA-Prompt, the strongest baseline, could not be reproduced by the authors, so InTAct was integrated only into CODA-Prompt.
Limitations
- Integrated only into CODA-Prompt; KA-Prompt integration not reproduced.
- Hypercubes from IA with per-neuron bounds can be loose and occupy unoccupied volume.
- Gains on DomainNet are small or absent (56.66 vs 58.66 for C-Prompt).
Experimental setup
Datasets: DomainNet ImageNet-R ImageNet-C ImageNet-Mix
Benchmarks: DomainNet ImageNet-R ImageNet-C ImageNet-Mix
Metrics: average accuracy (AA)
Baselines: EWC LwF L2P S-Prompts DualPrompt ESN CODA-P InfLoRA Cprompt C-Prompt KA-Prompt
Builds on: CODA-Prompt InterContiNet HINT L2P DualPrompt
Closest papers by meaning
- Mixture of Experts Meets Prompt-Based Continual Learning (NeurIPS 2024 · similarity 0.88)
- Continual Learning with Global Alignment (NeurIPS 2024 · similarity 0.88)
- A Unified and General Framework for Continual Learning (ICLR 2024 · similarity 0.88)
- Model Sensitivity Aware Continual Learning (NeurIPS 2024 · similarity 0.87)
- Learning without Isolation: Pathway Protection for Continual Learning (ICML 2025 · similarity 0.87)
- Train with Perturbation, Infer after Merging: A Two-Stage Framework for Continual Learning (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.